Getting Started with Azure IoT services: Event Hubs

Microsoft Azure Event Hubs is a managed platform component of Azure IoT services that provides a telemetry data ingestion at cloud scale with low latency and high reliability.

For your Internet of Things (IoT) scenarios, you can think of Event Hubs as the loosely-coupled beginning of an event pipeline that sits between event publishers like sensors and event consumers like Azure Stream Analytics. Unlike queues, Event Hubs implement partitions (shards) to support massive horizontal scale for the processing of millions of events per second. Consumer Groups provide consuming applications an independent view of the Event Hub from which to read the telemetry streams that can lead to complex event processing, storage or other downstream services.

Event Hub Graphic

Now that you have a brief summary of this event ingestion technology, it’s time to step through the creation of your own Event Hub so you can start bringing your IoT scenarios to life.

Go to your Azure Portal and click the Service Bus icon on the left side of the page as shown below:

Create Service Bus Namespace

If you have an existing Service Bus namespace, then you can reuse it. Otherwise, click Create a New Namespace.

The Create a Namespace dialog will pop up on your screen as shown below:

Create Namespace Dialog

In this dialog you will enter a unique Namespace Name, select a Region, select a Subscription to bill against, choose Messaging as the Type in order to support Event Hubs and choose Standard as the Messaging Tier. This allows you to support a sufficient number of Brokered connections (AMQP) into the Event Hub and up to 20 consumer groups leading out of the Event Hub.  Click the checkbox when you’re done.

With your Service Bus namespace created, click on the appropriate highlighted row as shown below:

Service Bus Created

Click Event Hubs from one of the choices across the top of the page to bring up the page shown below:

Create Event Hub

Click Create a New Event Hub.

Select Quick Create to which should be sufficient for most IoT scenarios.

Create Event Hub Quick Create

Enter a unique Event Hub Name, select the same Region as your Service Bus Namespace, select a Subscription to bill against, select the Service Bus Namespace you previously created and then click the Create a New Event Hub checkbox.

With your Event Hub created, click on the appropriate highlighted row as shown below:

Event Hub Created

Click Configure from one of the choices across the top of the page to bring up the page shown below:

EventH ub Configure

The Message Retention text box allows you to configure the number of days you’d like to have your messages retained in the Event Hub with a default of one day.  The Event Hub State combo box allows you to enable or disable your Event Hub.  Following the Quick Create path gave you a Partition Count of 16.  This value is not changeable once it’s been set so you might consider a Custom Create of your Event Hub if you need a different value.  Partitions refer to a scale unit where each one supports message ingress of 1 MB/sec and an egress of 2 MB/sec.  You can set the number of Event Hub throughput units on your Service Bus Scale page.  The default value is set to one.

In your next configuration step, you will create two shared access policies to facilitate security on your message ingress and egress as shown below:

SharedAccessPoliciesClick into the Name textbox and enter an ingress name then select the Permissions combo box and select Send.  Repeat the process on the newly created row below by adding an egress name and then select Manage, Send, and Listen from the combo box.  Click the Save icon at the bottom of the page and then you’ll notice that shared access keys are generated for both your message ingress and egress policies.  Those keys will be used to create the connection strings used by your IoT devices, gateways and event consumers like Azure Stream Analytics.

To view and use those connection strings, click Dashboard at the top of the page and then click the Connection Information key icon at the bottom of the page to bring up the Access connection information dialog as shown below:

Connection Strings

This is where you will go to copy the Shared Access Signature (SAS) key connection strings into your code to authenticate access to entities within the namespace. The authentication and security model ensures that only devices that present valid credentials can send data to an Event Hub. It also ensures that one device cannot impersonate another device. Lastly, it prevents a rogue device from sending data to an Event Hub by blocking it. Of course, all communication between devices and Event Hubs occurs over TLS.

To wrap things up, click Consumer Groups from one of the choices across the top of the page to bring up the page shown below:

Consumer Groups

Rather than using the $Default Consumer Group, it’s a good idea to specify one or more of them yourself to create views of the Event Hub that will be used by things Steam Analytics.  This is a simple process that starts with clicking the + Create icon at the bottom of the page.

The Create a Consumer Group dialog will pop up on your screen as shown below:

Parking Group

 

Type in a meaningful name in the Consumer Group Name textbox and then click the checkbox to save and exit.

Some of you may be wondering why do you need to use Event Hubs for event ingestion when you’ve been uploading data from disparate clients to servers using SOAP + XML and REST + JSON for more than a decade.  The answer has to do with wire protocol efficiency and reliability.  By default, Event Hubs use the Advanced Message Queuing Protocol (AMQP) which is an OASIS standard.  This is a binary, peer-to-peer, wire protocol designed for the efficient, reliable exchange of business messages that got its start on Wall Street.  If it’s good enough for the critical financial transactions between the world’s largest investment banks and stock exchanges, I’m pretty sure it’s good enough for the rest of us.

At this point, your Event Hub should be up and running. The next step is to get a device sending telemetry into your Event Hub so you can see it working. See you next time.

Rob

Increasing Revenue via the Internet of Things: Vending Machine Exact Change Alarm

How can the Internet of Things (IoT) help your company increase revenue?

Problem: The Exact Change light just illuminated on one of your vending machines. This reduces the number of potential customers of snacks or drinks to just those who are carrying exact change in their pocket. The revenue output of the vending machine has been cut and you’re making less money.

Solution: IoT

Remote telemetry is securely sent over the Internet from one of your vending machines to Azure Event Hub when the exact change light comes on. Reacting quickly is critical because time is money.

Vending Machine

Azure Stream Analytics pulls the data out of the Event Hub and runs a query against the streaming data in-flight. In milliseconds it determines the data contains an exact change alarm as well as the unique identifier of the vending machine.

Event Hub to Stream Analytics

A lookup query from Stream Analytics determines the exact location and route that the vending machine belongs to as well as the current route driver working the shift. All the appropriate data is inserted into an Azure SQL Database and an API call is made to Azure Notification Hubs.

Notification Hub

Azure Notification Hubs send a push notification to the route driver’s ruggedized handheld device to inform the mobile app that data needs to be retrieved from the Azure SQL Database.  The app utilizes Azure Mobile Services and therefore triggers a data sync action to download the information about the exact change alarm and location of the vending machine to the local SQLite database.

Mobile Services

The offline business data in the Universal App combined with offline Here maps allows the route driver to do his job and to navigate to the vending machine in the absence of network connectivity.  Upon arrival at the vending machine, the route driver adds nickels, dimes and quarters to the machine and the exact change light turns off.

Truck

IoT Value Prop: The vending machine switched from sitting idle with a limited potential customer base to a money maker for the business thanks to the Internet of Things and Microsoft Azure IoT Services.

This stuff doesn’t have to be complicated.

Rob

Increasing Revenue via the Internet of Things (IoT Example 8)

How can the Internet of Things (IoT) help your company increase revenue?

What’s better than just reacting to a failed refrigeration compressor in one of your vending machines? Repairing it before it fails so revenue and customer satisfaction aren’t adversely impacted.  Failures of various hardware components are an unfortunate part of many businesses.  Yearly revenue losses due this is often anticipated and “baked-in” to a company’s financial numbers.  Streaming operational data about a vending machine’s refrigeration compressor to the cloud and utilizing machine learning is a great way to disrupt “business as usual.”  This telemetry is fed into dynamic, predictive models, so failures can be anticipated and repairs can be make before a breakdown occurs.  Revenue losses turn into revenue increases.

Other IoT scenarios:

Increasing Customer Satisfaction via the Internet of Things (IoT Example 2)

Increasing Revenue via the Internet of Things (IoT Example 7)

Increasing Customer Satisfaction via the Internet of Things (IoT Example 1)

Increasing Revenue via the Internet of Things (IoT Example 6)

Reducing Expenses via the Internet of Things (IoT Example 4)

Increasing Revenue via the Internet of Things (IoT Example 5)

Increasing Revenue via the Internet of Things (IoT Example 4)

Reducing Expenses via the Internet of Things (IoT Example 3)

Increasing Revenue via the Internet of Things (IoT Example 3)

Increasing Revenue via the Internet of Things (IoT Example 2)

Reducing Expenses via the Internet of Things (IoT Example 2)

Increasing Revenue via the Internet of Things (IoT Example 1)

Reducing Expenses via the Internet of Things (IoT Example 1)