Geo Spatial Data analysis using WSO2 Complex Event Processor

person kasunfolder_openUncategorizedaccess_time December 12, 2014

Introduction

Today we are living in a fast moving world. Humans are rapidly moving forward with the technology and also in their real physical lives. Keeping track of those movements have become critical and important for regular people and as well as for the enterprises. There are lots of ways to get location data about an object on the speare such as through GSP satellites and Cellular Positioning Techniques via GSM Networks. But just a collection of data does not have any real value unless we process those data to generate meaningful information. And it creates a new question regarding how we can process huge amounts of spatial data generated by location device. And that is where WSO2 complex event processing(CEP) comes into play.

Figure 1

For precise measurements, calculations or to display spatial data , we need to have them in real time. However the problem is that real time events coming from large number of devices generate huge amounts of event stream. WSO2 CEP is capable of analysing data coming from different types of streams at a high rate and produce useful information out of those data. This article explains how we can process spatial data coming from an external source of events and analyse/manipulate those data to produce meaningful information to end users, while hiding the complex CEP process by WSO2 Jagger UI application.

Features Available in Geo-Dashboard

WSO2 Geo-Dashboard is based on three previous projects which were done as proof of concepts. Geo-Dashboard comprises of features in those three projects with new CEP query structure allowing the addition of dynamic geo operations with some UI improvements to minimize exposing users from CEP configurations.

Show real time spatial data on the map(with object rotations)

With CEP Geo-Dashboard, users can see the movements of spatial objects in real-time, including their rotations on the space.WSO2 CEP accept any type of input data through its supported input adaptors (Figure 2).

Figure 2
Figure 2

Events which are coming through an input adaptor go through an event builder to map the data attributes in the specific event format to siddhi event attributes to process them by Siddhi engine. For the demonstration purpose in this article we have used HTTP input adapter to collect events which are sent from the sample data producer in JSON encoded format, then we put them into rawInputStream for further processing. After processing the event by siddhi engine(discussed later in the article), we format the event into our desired output format, and output the event through an output adaptor.

In the CEP Geo-Dashboard, spatial object data is transferred as a GeoJson[] object from the CEP websocket output adapter to the web browser. This allows us to loosely couple the front end application with CEP. A marker on the map is placed using GeoJson point which is received from the CEP and keep an array of spatial objects to update the location as the events come in. We have used properties member of the GeoJson structure to send custom attributes such as speed, heading, state and information data. We have used Leaflet[] javascript framework to handle map data on the browser.

Figure 3
Figure 3

Speed alert

Speed alerts are the most simplest form of alerts currently  we have. Users can set a specific value (in ms-1) for the speed limit and it will be set as a global value for all the spatial events passing through the CEP. Following figure shows an example of a CEP siddhi query used to trigger speed alerts.

from dataIn[speed >= 34]#transform.geo:subscribeExecutionPlan()

select id , latitude, longitude,timeStamp, speed, heading ,eventId , “ALERTED” as state, “This vehicle is going over speed!!” as information

insert into dataOut;

from dataIn[speed < 34]

select id , latitude, longitude,timeStamp, speed, heading ,eventId , “NORMAL” as state, “Normal driving pattern” as information

insert into dataOut;

Figure 4

We have used XML templates of the execution plans as a convenient way of keeping the structure of the execution plans, rather than type them in the source itself. This templating mechanism will be discussed in detail later in this article.

Proximity alert

Proximity alerts are triggered when two or more objects (vehicles) become close in their proximity by predefined value from the user. When two or more objects become closer in proximity based the user defined value, we change the state attribute of the event to ALERTED , which notifies an alert in the web app UI.

Figure 4
Figure 4

Users can define a proximity zone by drawing a circular area centering an object and assign a time constraint for how long should the second object need to keep in close proximity in order to trigger an alert. Similarly speed alert also applies globally where if any two or more objects become closer than the user defined value , an alert will be triggered from the CEP.

from dataIn#transform.geo:subscribeExecutionPlan()

select id, latitude, longitude, timeStamp, speed, heading, eventId,geo:geoProximity(50,latitude,longitude,id,timeStamp,20) as proximity

insert into innerStream;

from innerStream[proximity == “false”]

select id , latitude, longitude,timeStamp, speed, heading ,eventId , “NORMAL” as state, “” as information

insert into dataOut;

from innerStream[proximity != “false”]

select id , latitude, longitude,timeStamp, speed, heading ,eventId , “WARNING” as state, proximity as information

insert into dataOut;

Figure 5

Stationary alert

Stationary alerts are used to notify the users about the state changes in a predefined geo stationary. If a device stays within a predefined buffer-zone stationed for a predefined period of time, an alert will be generated and it will be displayed as shown below.

Figure 6
Figure 6

Stationary alerts are triggered using geo-withinbuffer extension in CEP. It checks for a particular event location with the defined stationary zones and check whether the current event is located within the defined zone. If that is true then it calculates the time spatial object have been staied in the defined stationary zone and if both become true, then we change the event state to ALERTED and append the appropriate information about the alert. Following WSO2 CEP execution plan has been used to trigger stationary alerts from the incoming spatial events.

from dataIn#transform.geo:subscribeExecutionPlan()

select id, latitude, longitude, timeStamp, speed, heading, eventId,geo:withinstationery(longitude,latitude,“{‘type’:’Point’,’coordinates’:[79.87062692642212,6.883652362379861],’radius’:360.8415505794831}”,id,timeStamp,3) as isStationery

insert into innerStream;

from innerStream[isStationery != true]

select id , latitude, longitude,timeStamp, speed, heading ,eventId , “NORMAL” as state, “” as information

insert into dataOut;

from innerStream[isStationery == true]

select id , latitude, longitude,timeStamp, speed, heading ,eventId , “ALERTED” as state, “This vehicle is in Harbour_bus_stop area!!!” as information

insert into dataOut;

Geo-fence Alerts

Geo fencing allows users to set up triggers such that ,when a device enters or exits the defined geometry boundaries defined by the user, it pops up an alert in the dashboard. Geo fencing has many practical uses. For example, a school administrator can set up alerts so when a school laboratory tablet device leaves the school premises, the administrator can disable the device. A marketer can geofence a retail store in a mall and send a coupon to a customer who has downloaded a particular mobile app when the customer (and his smartphone) crosses the boundary.

Figure 7
Figure 7

The algorithm used behind the geo-fencing operations is, we convert the user drawn area to geoJson format and input it as a constant to CEP geo-fence custom extension. When an event is received to the CEP, its coordinates are checked against the user defined fence for whether it is within the defined boundaries or not, using  OSGeo GeoTools library. The following sample query shows the usage of geo-fencing custom extension (iswithin) in siddhi queries.

from dataIn[geo:iswithin(longitude,latitude,“{‘type’:’Polygon’,’coordinates’:[[[79.8564863204956,6.947045831920686],[79.85642194747923,6.946747629898817],[79.8564863204956,6.947045831920686]]]}”)==true]#transform.geo:subscribeExecutionPlan()

select id , latitude, longitude,timeStamp, speed, heading ,eventId , “ALERTED” as state, “This vehicle is in Kotahena_area restricted area!!!” as information

insert into dataOut;

from dataIn[geo:iswithin(longitude,latitude,“{‘type’:’Polygon’,’coordinates’:[[[79.8564863204956,6.947045831920686],[79.85674381256102,6.948643339531811],[79.85577821731567,6.945128815623514],[79.85601425170898,6.945533519714717],[79.85620737075806,6.94600212401683],[79.85629320144653,6.946342926852387],[79.85642194747923,6.946747629898817],[79.8564863204956,6.947045831920686]]]}”)!=true]

select id , latitude, longitude,timeStamp, speed, heading ,eventId , “NORMAL” as state, “” as information

insert into dataOut;

Add external tile URLs and Web Map Service(WMS) endpoints

Slippy map and Tiles

The slippy map is an Ajax component. JavaScript runs in the browser, which dynamically requests maps from a server in the background (without reloading the whole HTML page) to give a smooth slippy zoomy map browsing experience. The implementation of this is provided by a javascript library Leaflet. The map image is built up of many little square images called “tiles”. These are rendered and served from a “tile server”.

A tile server typically does not render tiles in real time for each user browsing the map. The tiles are rendered ahead of time and stored on the disk. Even so, relatively few rendering programs have a proven track record for serving high traffic regularly updating worldwide maps.

Figure 8
Figure 8

There are different tile servers which are specialized for rendering geographical information on a particular area or place such as OpenCycleMap and OpenSeaMap. In Geo-Dashboard we can add new tile-servers so that users can see the maps with different detail levels. The default tile server for the dashboard is openstreetmap. This feature allows the users to add dynamic map layers to the dashboard allowing them to show spatial objects on top of their desired map layer.

Web Map Service

A Web Map Service (WMS) is a standard protocol for serving georeferenced map images over the internet that is generated by a map server using data from a GIS database. The response to the request is one or more map images (returned as JPEG, PNG, etc.) that can be displayed on a browser and desktop applications. Geo-Dashboard has a feature to add new WMS endpoint, so that users can see their spatial objects on top of a given MWS service map layer. For example, if a corporate user already has a web map service which serves their GIS data through WMS service, they can add the WMS endpoint to geo-dashboard and locate their spatial objects (i.e. Vehicles, People, Mobile devices, etc.) on top of their private WMS layer.

Technology used with Geo-Dashboard

WebSockets

The WebSocket Protocol is an independent TCP-based protocol that uses an HTTP friendly Upgrade handshake. WebSocket connections are full-duplex, long-lived and bi-directional. After the initial handshake, there is no transactional semantics and there is very little per message overhead. The client and server may send messages at any time and must handle message receipt asynchronously.

Websocket input and output adapters are available with wso2 CEP (>= 4.0.0). I have used websocket output adapter that comes with the CEP to push the data from CEP to Geo-Dashboard as shown in Figure 1. It helps to keep the entire system idled if there is no incoming event and also websocket eliminates the unnecessary network usages where in AJAX long polling mechanism. Output data structure of the CEP websocket is a Geojson which is the commonly used standard for transporting spatial data. By taking the processed and fused events(Events fusion will be discussed later in this article) from the FusedSpatialEvents(Figure 9) stream and have built the Geojson by the CEP output even formatter in JSON, finally formated events are pushed through the CEP websocket output event adapter to the client side application (JavaScript websocket).

Figure 9
Figure 9

Leaflet JS web framework

Leaflet is a commonly used open source JavaScript library for manipulating and projecting map data on web-browser. In geo-dashboard we have leaflet core library for putting markers on the map, appling tile and WMS layer and rotating markers and lot more, other than the core library I have used some Leaflet plugins to create attractive user experience. Such as leaflet.draw for drawing and editing vectors and markers on Leaflet maps, leaflet.groupedlayercontrol for add groupings to overlay layers object, and swap out the default layer control with the new one and leaflet.MeasuringTool to measure the distance of a path. And also to measure the distance between multiple points.

Architecture and Design

WSO2 CEP configuration/Extensions for Geo-Dashboard

Information displayed on Geo-Dashboard are flown through through the CEP and those information are generate by CEP from the incoming events from the external geospatial applications.WSO2 CEP is used for manipulate those events and extract information from them and finally push those information to front end application(Geo-Dashboard) to represent them to end-users.Because of that geo-dashboard is highly depend on WSO2 product CEP.

WSO2 CEP has large variations of input data types  which calls Input Event Adaptors,such as email,file,http,jms,kafka,mqtt,etc., In Geo-Dashboard application I have used HTTP input adaptor to take events into the CEP and then use two event builders to map incoming event data attributes to CEP stream attributes.As shown in the figure 10 I have use geoJsonBuilder and TrackerJsonBuilder to build a rawInputStream event.geoJsonBuilder is used to extract data from GeoJson encoded HTTP message where GeoJson is a common format for encoding a variety of geographic data structures.and TrackerJsonBuilder is used for extract data from anyother json encoded HTTP message.Even tho I have use HTTP Input Event Adapor and JSON data type for input data into CEP , it is not limited to input data only by that combination of Input Event Adapor and Event Builder.

Figure 10
Figure 10

In CEP configuration for Geo-Dashboard I have allowed upto inputStandardizer component to dynamically change the configurations in CEP, which means users can use any Input Event Adaptors and Event Builders and any data type and number of attributes for rawInputStram. inputStandardizer query converts the raw data into standard event stream where later down in the event flow used to take events from it for process the spatial data.

Input Standardizer

from dataIn

select id, latitude, longitude, timeStamp, speed, heading, geo:generateEventId() as eventId

insert into dataOut

In standardizing query we take events from rawInputStream and give an UUID as eventId attribute for every event to recognize each event uniquely when there are multiple copies of events generated for single input event.Multiple events are generated because of taking single events by multiple siddhi queries(execution plans) and insert them into same output stream where there slightly change with the original event is inserted into the output stream.

This scenario is shown in figure 11.

Figure 11
Figure 11

Events in the standardSpatialEvents stream suppose to have recommended minimum data for uniquely identify and locate event data on a map.A sample definition of the standardSpatialEvents is shown in figure 12. Execution plans which imports the standardSpatialEvents can guarantee to have set of mandatory data attributes from the events on that stream.

{

  1. “name”:“standardSpatialEvents”,

  2. “version”:“1.0.0”,

  3. “nickName”:“”,

  4. “description”:“Standardized event by spatial InputStandardizer execution plan”,

  5. “payloadData”:[

    1. {

      1. “name”:“id”,

      2. “type”:“STRING”

    2. },

    3. {

      1. “name”:“latitude”,

      2. “type”:“DOUBLE”

    4. },

    5. {

      1. “name”:“longitude”,

      2. “type”:“DOUBLE”

    6. },

    7. {

      1. “name”:“timeStamp”,

      2. “type”:“STRING”

    8. },

    9. {

      1. “name”:“speed”,

      2. “type”:“FLOAT”

    10. },

    11. {

      1. “name”:“heading”,

      2. “type”:“FLOAT”

    12. },

    13. {

      1. “name”:“eventId”,

      2. “type”:“STRING”

    14. }

  6. ]

}

Figure 12

Dynamically adding custom extensions(GeoSpatial functions)

Execution plan templates

Most of the actions user performs in the GeoDashboard UI effect a change in back-end configurations, for example if user sets a new speed alert value in GeoDashboard UI(Figure4) it triggers a SOAP call to WSO2 CEP services/EventProcessorAdminService.EventProcessorAdminServiceHttpsSoap12Endpoint/ endpoint with editActiveExecutionPlanConfiguration XML encoded data(sourcecode) and SOAP message body contains the new execution plan XML in urlencoded form.This action cause to redeploy an existing execution plan with changed value for speed alert in CEP.

Sending new execution plans to CEP via SOAP messages is achieved with execution plan templates.Execution plan XML are stored as template(i.e: speed_alert template) within the jaggery application, and dynamically changing fields in the execution plan are stated as keys in the execution plan template prefix with ‘$’ character, so that when users has made a change in GeoDashboard front-end jaggery application those changed values in execution plan are replace with the encoded key values in the template and finally the new execution plan with the parsed values with the relevant key in the template file is send as SOAP message to WSO2 CEP, which then re-deploy the given execution plan with the new execution plan XML  received via SOAP message.

Fusing redundant events

Because of the design and the configuration of the event flow in WSO2 CEP for geoDashboard , multiple execution plans can be added pralla where every query takes events from standardSpatialEvents  stream and insert into processedSpatialEvents stream , this flexible feature allows the users to add any number of execution plans dynamically to CEP for add geospatial functionality.For example, GeoDashboard allow users to define any number of geo-fence areas on the map using give UI tools, and each of those geofence area is added to CEP as an execution plan to check the spatial object which are located inside the fence area, this was achieved by parallely appending geo_within_{query_name}_alert execution plans between standardSpatialEvents  stream and processedSpatialEvents stream

This flexibility introduce new issue that, parallel execution plans(Siddhi queries) create a copy of the original  event which was taken from the standardSpatialEvents  stream.

Figure 13
Figure 13

We have use a registration mechanism that is, when deploying an execution plan , at its initialization method we upcount a counter where its value is used to fuse the redundant events into one,this method is shown in figure 14.

public InEvent eventsFuser(InEvent event) {

String[] statesArray = new String[]{“OFFLINE”, “NORMAL”, “WARNING”, “ALERTED”};

List<String> states = Arrays.asList(statesArray);

Object[] data = event.getData();

String finalState = “”;

String information = “”;

String eventId = (String) event.getData(variablePosition);

ArrayList<InEvent> receivedEvents = eventsBuffer.get(eventId);

String alertStrings = “”;

String warningStrings = “”; // TODO: what if no warnings came ?

Integer currentStateIndex = –1;

for (InEvent receivedEvent : receivedEvents) {

String thisState = (String) receivedEvent.getData(7);

Integer thisStateIndex = states.indexOf(thisState);

if (thisStateIndex > currentStateIndex) { // TODO: this and current little bit confusing??

finalState = thisState;

currentStateIndex = thisStateIndex;

}

if (thisState.equals(“ALERTED”)) {

alertStrings += “,” + (String) receivedEvent.getData(8);

} else if (thisState.equals(“WARNING”)) {

warningStrings += “,” + (String) receivedEvent.getData(8);

}

}

if (finalState.equals(“NORMAL”)) {

information = “Normal driving pattern”;

} else {

if (!alertStrings.isEmpty()) {

information = “Alerts: ” + alertStrings;

}

if (!warningStrings.isEmpty()) {

information += ” | Warnings: ” + warningStrings;

}

}

Object[] dataOut = new Object[]{

data[0], // id

Double.parseDouble(data[1].toString()), // Latitude

Double.parseDouble(data[2].toString()), // Longitude

data[3].toString(), // TimeStamp

Float.parseFloat(data[4].toString()), // Speed

Float.parseFloat(data[5].toString()), // Heading

eventId,

finalState,

information

};

return new InEvent(event.getStreamId(), System.currentTimeMillis(), dataOut);

}

Figure 14

Precedence of states

Precedence of states goes low to high from LHS to RHS
OFFLINE < NORMAL < WARNING < ALERT
Higher the index higher the Precedence, States are in all caps to mimics that states not get changed with the time.In the event fusion method (figure 15) we check the state of the events and assign a final state to the fused event based on their precedence,the final state of the event combining the results of redundent events produce by the parrel execution plans is assign by the precedence of the state.This state attribute in the event is then used by the GeoDashboard to visualise the marker with different colors for different states(SourceCode).

Geo-Dashboard user interface design

Most of the GeoDashboard UIs are made up with bootstrap which is the most popular HTML, CSS, and JS framework for developing responsive, mobile first projects on the web.And also I have use UiKit HTML,CSS and JS framework for offcanvas sidebar bar for navigating through components.For handling spatial data and map data I have use an awesum JS plugin called Leaflet with some plugins to enhance the user experiences.HTML pages are rendered from Jaggery framework engine.

Single Page Application(SPA) Concept

A single-page application (SPA), is a web application that fits on a single web page with the goal of providing a more fluid user experience akin to a desktop application. In a SPA, either all necessary code  HTML, JavaScript, and CSS is retrieved with a single page load, or the appropriate resources are dynamically loaded and added to the page as necessary, usually in response to user actions. In GeoDashboard core components are loaded from the root file(index.jag) such as global map,offcanvace sidebar, about, privacy etc,and all the geo spatial functions are loaded to main page by remote ajax calls when the user perform the action.This separation of functions made it easy to add new functions to GeoDashboard.

The page does not reload at any point in the process, nor does control transfer to another page.

Single login authentication

When a user login to GeoDashboard using the tenant credentials their login information is stored in browser cookie(Sourcecede) for authentication when the need to authenticate again.

Configure and setup Geo-Dashboard

Following list of steps shows the instructions to how to setup WSO2 CEP to run Geo-Dashboard application,In brief you will need to download product CEP and Jaggery(Unless CEP does not contains a jaggery server itself),and then you have to clone the Geo-Dashboard repository from github and place it under {jaggery.home}/apps/ directory to load it as a jaggery app when the server bootup next time.

Geo-Dashboard required some CEP extensions to be deployed on WSO2 CEP in order to manipulate spatial data going through wso2 cep, those extensions can be found from here.

Geo-Dashboard configuration-wizard has made it easy to up and running samples quickly with few click.

Sample data sets for demonstrate the geo-dashboard can be found in here.

  1. Get the latest CEP stable build from here
  2. Get the latest jaggery build from here
  3. Get the geo-dashboard app from github repo and copy app directory to {jaggery.home}/apps/
  4. Get all the geo-extensions from here or here(github)
  5. Build those extensions(or get already build extensions from here) and put into {cep.home}/repository/components/lib/
  6. Add following Full qualified class names to (FQCNs) to {cep.home}/repository/conf/siddhi/siddhi.extension file

    org.wso2.cep.geo.notify.NotifyAlert

org.wso2.cep.geo.functions.EventIdGenerator

org.wso2.cep.geo.libs.FuseEvents

org.wso2.cep.geo.libs.executionPlanSubscriber

org.wso2.cep.geo.proximity.GeoProximity

org.wso2.cep.geo.GeoIsWithin

  1. Get GPS sample data producer(you need to have oracle java 1.7 installed in you machine in order to run the sample data producer)
  2. Set port offset value to 1 in cep server and use that port number in following places accordingly
  1. Jaggery cep_info.json file (Or run the setup-wizard, it will create the correct configuration files)
  2. JavaScript application_options.js file
  3. GPS sample data producer (6th step) DataAgent/src/com/client/Main.java [Line:10]
  1. Change the database.json files according to  your MySQL database environment (Or run the setup-wizard)
  2. Run the MySQL database setup script (geo_db_setup.sh) to configure MySQL database to store alert toggle notifications (Get JDBC Driver if required.)
Figure 15
Figure 15
  1. Download CEP configuration files (eventbuilders,eventformatters,executionplans,inputeventadaptors,outputeventadaptors) and copy into {path_to_cep}/repository/deployment/server/ directory.
  1. Download CEP stream definitions and append to {path_to_cep}/repository/conf/data-bridge/stream-definitions.xml file.
  2. If you get the following error when trying to login, Most probably it is cause by using outdated jaggery server.

HTTP Status 500 – org.mozzila,javascript.EcmaError: TypeErrorL Cannot find tenantUser in object [object Object].(/geo_dashboard/controllers/authentication.jag#13)

Copy files in this repository to your local

{path_to_jaggery_server}/modules/carbon/scripts/server/ directory in Jaggery server.

Assuming you have successfully configured above steps.

  • Start the Jaggery App with the default port and run the geo app through jaggery management console.

(eg: http://localhost:9763/geo_dashboard/)

  • Login to the app providing admin as the username and password(Same Username password used for login to CEP).
  • Start the CEP with the offset and login using admin credentials(admin)
  • Run an ant on sample client mentioned above.

Please find below a sample event flow which will display after you have successfully configured the CEP.

Figure 17
Figure 17

Further works

Implement Minimum bounding rectangle(MBR) algorithm to find nearby spatial object in real time.

Conclusion

This article covers information about  the newly introduced spatial extensions to WSO2 CEP and Jaggery dashboard  for displaying the geo-spatial data processed through the CEP. Information regarding the purpose and benefits of these extensions, how to configure new extensions with WSO2 CEP,and architectural design is included in this article.

I have discussed about how to extend the existing GeoDashboard user interfaces to add new spatial features to front-end of the Geo-Dashboard and how to add new spatial query parallel to the currently deployed extensions in WSO2 CEP.In addition, CEP event streams are also re-defined to facilitate common spatial data flows.

This sample could be used to analyse scenarios where we get streams of GPS data from fleets of moving devices.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code class="" title="" data-url=""> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong> <pre class="" title="" data-url=""> <span class="" title="" data-url="">