Smart Meter Data Analytics with Splunk – Part 2

In last week’s post we detailed the Glowstick that provides a bridge between your Smart Meter and your wifi network. Your energy and gas usage is logged to a cloud service which can be accessed via an API. In this post we continue to show you how you can perform Smart Meter Data Analytics with Splunk.

Glowmarkt API

The Glowmarkt API is documented here. Prior to accessing the the data endpoints an OAuth must be performed. In a future blog post we will detail how this can be coded using python and the Splunk Add-On Builder.

Glowmarkt Add-On for Splunk

We have released an initial version of an add-on that connects to the Glowmarkt API and pulls through a live electricity usage reading. Further development is planned to also retrieve gas usage.

Once you have installed the Glowmarkt device and setup an account, you can download the add-on and configure it to pull your energy data into Splunk. The add-on is available from Splunkbase here.

Add-on Configuration

Once the add-on is installed your credentials (setup using the Bright smartphone app) should be added via the Configuration tab. Click Add and then input your username and password.

Then from the Inputs tab Create New Input should be clicked. Most of the options are self-explanatory. The default for applicationId should be left as is, leaving resourceId as the remaining option to complete. A future version of the Add-on will retrieve this programmatically but for now consult the API documentation and use either a curl command or the provided postman collection to retrieve the Electricity Consumption resourceId.

Smart Meter Data Analytics

Now we have the data in Splunk we can start to write some searches. This early version of the add-on writes the data received from the API straight through without modification. The “data” payload of the API response contains two values in an array, one representing the timestamp of the event and the the other the live electricity consumption at that time. A future version of the add-on will write a separate field to represent the live reading and the timestamp value will set the event time.

Visualising the data

With the data in it’s current form we need to pull the second value from the data array, this is the live meter reading in Watts. We convert this to kilowatts and then use the timechart command to tranform the data for visualisation. We keep the span at 30s so that each data point represents a reading from the API (1:1). No actual aggregation should therefore occur.

index=main sourcetype="glowmarkt:electricity:current" 
|eval elecLive=mvindex('data{}{}',1)
|eval elecLiveKw=elecLive/1000
|timechart span=30s max(elecLiveKw) as elecLiveKw

This produces the following timechart:

This data can also be used for a single value visulisation, as shown below.

Alert Search

The search below can be setup as an alert search to warn you if consumption stays high over a prolonged period of time. While this is just scraping the surface of the potentially useful analytics that can be performed it is useful way to let Splunk keep an eye on your energy usage.

index=main sourcetype="glowmarkt:electricity:current" earliest=-15m
|eval elecLive=mvindex('data{}{}',1)
|stats avg(elecLive) as avgWatts

The screenshots below show an example alert setup, tune the values to suit. If you want to alert if your usage stays above 3kw for 10 minutes then change the custom trigger to be “search avgWatts > 3”.

• 

• 

Conclusion

With this initial setup some useful analytics and alerting can be performed. In future posts we will look at extending the analytics further. Subscribe to the mailing list to ensure you get notified when they are posted.