Storm-Streaming

This is a sample storm topology for stream processing sensor data. In this case, we're applying simple rules to events being generated by a trucking fleet. The goal is to send email alerts to a dispatch center when a threshold of "Un-Safe" events have been generated by a specific truck/driver.

Building

There are three dependencies to building this example:

1. Java 1.6
2. Maven
3. ActiveMQ
4. storm-jms

Dependencies 1, 2, and 3 can be downloaded and installed respectively. Once they've been installed and you can do a mvn -version from a command prompt, I'll cover how to build and install storm-jms, and then this project.

Building Storm-JMS

I've forked @ptgoetz's storm-jms project, as one addition was needed to build the project. A pull request has been issued and once it has been added, you can build directly from his repo. For now, we'll go ahead and get it from my repo using the commands below:

git clone https://github.com/pcodding/storm-jms.git
cd storm-jms
mvn clean install

Building Storm-Streaming

Now that the storm-jms dependency has been installed, we can build this project. The process is very similar and is as follows:

git clone https://github.com/pcodding/storm-streaming.git
cd storm-jms
mvn clean install

Project Explanation

This project uses the following classes to define the topology com.hortonworks.streaming.impl.topologies.TruckEventProcessorTopology. It has one spout, JmsSpout, and one bolt, TruckEventRuleBolt. The TruckEventRuleBolt class receives each Tuple, and processes it using the TruckEventRuleEngine class. The TruckEventRuleEngine class stores each "Un-Safe" event for a driver and sends an email if 5 events have been captured.

Project Configuration

The src/main/resources/config.properties has the following properties:

[email protected]
notification.subject=Unsafe Driving Alerts
jms.host=sandbox
jms.port=61616
jms.notificationQueue=sensor_data
jms.notificationTopic=sensor_events

The notification.* are related to the emails sent after the event threshold has been reached, and the jms.* configuration properties control the JMS connection and system details. Make sure you edit the notification.email before running the project.

Running the Project

To run the project, just use the run.sh shell script to compile and execute the project on a stand-alone machine.

Simulating Data

What's a Storm topology without events? Not too exciting. I've created a data simulator that helps with simulating unsafe driving events among other things. You can find it here, and this is the process to get started:

git clone https://github.com/pcodding/stream-simulator.git

Edit the src/main/resources/config.properties to point to your ActiveMQ installation, then build and run the project:

mvn clean package
./run.sh 6 -1 com.hortonworks.streaming.impl.domain.transport.Truck com.hortonworks.streaming.impl.collectors.JmsEventCollector