OAuth with the Splunk Add-on Builder

in this article we’ll show how you a step-by-step approach to performing an OAuth authentication when developing in the Splunk Add-on Builder.

In a previous post we introduced the GlowStick Zigbee CAD, a device that collects your smart meter data and makes it available to you via an API.

Understand the Journey

The API you are trying to code against should of course have documentation, perhaps even “curl” examples of how to perform the OAuth journey. Read this and try out the “curl” examples with your credentials., make sure you can see that a token is returned in the response.

Use Postman

Postman is an API platform for building and using APIs. One key feature is that it will generate the code required to call the API that you are coding against. Through the user interface you can add the URI of the API, any headers required and the payload.

As indicated above, we can use Postman to provide code that makes the API call. There are various language and library options available, for the purposes of Splunk Add-on development we will use the Python – Requests library. Clicking on the “Code” link as shown in the screenshot above provides the code, as shown below.

Use Add-on Builder to Create the App

Use the Splunk Add-on Builder to create your app. To get started click “New Add-on”:

• Note the Splunk Naming Conventions for Apps and Add-ons
• Set the Add-on name, author, colour scheme and upload an icon if you have one

Create Data Input

The next step is to start the setup of a data input. You will need to know the parameters required to complete the OAuth authentication along with any other parameters required to fetch the data once the authentiation is complete. This may be as straightforward as a a client_id and secret but some scenarios may require more. By completing step 1 “Understand the Journey” you should already have an understanding of this.

• Click “Configure Data Collection”
• On the “Data Input Properties” tab:
  • Click “New Input”
  • Click “Modular input using my Python code”
  • Supply values for the data input properties:
    • Source type name = the splunk sourcetype of the data the input will collect
      • example = glowmarkt:electricity:current
    • Input Display Name = The display name of the input within the Add-on
      • example = Electricity Consumption
    • Input Name = the internal name of the input
      • example = electricity_consumption
    • Description = a description for the input
    • Collection Interval = how frequently to collect data
      • Example = 30 seconds (make the API call to collect data every 30 seconds)
  • On the “Data Input Properties” tab
    • This is where you will configure the parameters that the end user of the add-on will enter. The “Global Account” allows you to setup a username/password (or client_id/secret in OAuth context) that can be used by all inputs configured in the add on
    • Parameters that are required per input are also added here
      • for the glowmarkt API we require an applicationId and a resourceId to make the data call
      • drag a text input over to setup an input parameter for the API call, one to reuest the applicationId and another to request the resourceId
      • drag over the Global Account option to add a drop down so that the user can select which Global Account to use
  • Click Next

Supply the Code

Replace the supplied python code with the following:

import os
import sys
import time
import datetime
import json
import utils.auth as rauth
import utils.utils as utils
     
def validate_input(helper, definition):
	pass
	
def collect_events(helper, ew):
	pass

Now we need to add some code into the correct directory within the Add-on:

• Create a directory: ${SPLUNK_HOME}/etc/apps/<TA-NAME>/bin/utils
• In the new directory create an empty file: __init__.py

auth.py

The file auth.py is where we will perform the OAuth authentication and retreive the token to be used on the subsequent call to collect the data.

• In the utils directory, create another file : auth.py
  • in auth.py we will create a method get_access_token that will perform the Oauth and return the access token
  • in this get_access_token method you will need to codespecifically againt the OAuth endpoint for your scenario
  • the code we generated in the “Use Postman” section will help here
  • note that this API needs the username and password in a specific format

# encoding = utf-8
import sys
import json
import requests

def get_access_token(username, password, application_id):
    url = "https://api.glowmarkt.com/api/v0-1/auth"
    payload="{\"username\": \"%s\", \n\"password\": \"%s\"\n}"%(username,password)
    headers = {
        'Content-Type': 'application/json',
        'applicationId': application_id
        }
    try:
        response = requests.post(url, headers=headers,data=payload).json()
        return response['token']
    except Exception as e:
            raise e

utils.py

Now we create another file, utils.py. The code in this file will call the API to retrieve the data. You could use Postman again to help ensure you have the correct headers and to check out what the response looks like.

# encoding = utf-8
import sys
import json
import requests

def get_items(helper, access_token, url, applicationId):
    header = {'Content-Type': 'application/json',
    'token': access_token,
    'applicationId': applicationId}

    try:
        r = requests.get(url, headers=header)
        r.raise_for_status()
        response_json = json.loads(r.content)
        
    except Exception as e:
        raise e

    return response_json

Back to the Builder

Switch back to the Add-on Builder and click on Add-On Setup Parameters.

• Supply the username and password (client_id, secret) required to make the OAuth call
• Click on the Data Input parameters and supply any parameters you defined as required

Now we need to supply the code for the collect_events method we can see in the Add-on builder Code Editor:

import os
import sys
import time
import datetime
import json
import utils.auth as rauth
import utils.utils as utils
     
def validate_input(helper, definition):
	pass
	
def collect_events(helper, ew):
    global_account = helper.get_arg("global_account")
    username = global_account["username"]
    password = global_account["password"]
    applicationid = helper.get_arg("applicationid")
    resourceid = helper.get_arg("resourceid")
    
    access_token = rauth.get_access_token(username, password, applicationid)
    
    if(access_token):
        
        helper.log_debug("Calling glowmarkt API")
        url = "https://api.glowmarkt.com/api/v0-1/resource/%s/current" % (resourceid)
        electricity_reading = utils.get_items(helper, access_token, url, applicationid)
        
        event = helper.new_event(
            data=json.dumps(electricity_reading),
            source=helper.get_input_type(), 
            index=helper.get_output_index(),
            sourcetype=helper.get_sourcetype())
        ew.write_event(event)

    else:
        raise RuntimeError("Unable to obtain access token. Please check the username, password and applicationId")

Hit Save then Test. If everything is in place then you should see the JSON output of the API call in the right hand side. Congratulations on coding an OAuth journey with the Splunk Add-on Builder!

Validate your Add-on

There are more inputs to add to this add-on but we are going to skip to the end for now. Prior to uploading to Splunkbase you should validate your add-on. Even if you don’t plan to upload to Splunkbase it’s a good time to check that everything is in order.

As you can see below the Add-on passed validation with one single warning about their being no events. We haven’t setup any inputs in the actual add-on, the reason for this is so that the downloaded package doesn’t contain any user config.

Finally you can click on the “Download Package” button to get your newly created app and share it with your friends!

Credits and Thanks

In 2019 I was fortunate to be invited to and attend the Splunk Partner Technical Symposium in Berlin. Splunk had arranged for a number of great speakers to give proper deep dive technical talks on a variety of subjects. One of these talks explained how to perform an OAuth authentication while using the add-on builder to generate a new add-on. Many thanks to Jason Conger for this session. You can access his notes and code on github here.