If you are building or running an enterprise Splunk installation then you’ll likely have a bunch of SSL certificates to create or manage. Enterprises will generally have their own internal Certificate Authority, often with different signers for production and non-production environments.

To configure Splunk in a secure manner requires setting up a number of ports with SSL. Working through the required configuration in your local dev lab (using virtual machines or cloud instances) is a good way to become familiar with the process, validating you have the right options configured. It can also serve as a good baseline to compare against if you have any issues with a production build.

In order to do this, you will want to be able to generate your own certificates. There are various ways to do this but Minica is probably one of the easiest.

Minica is written in Go, so you’ll need to get that installed first if you don’t have it already. You can download Go from here: https://golang.org/dl/

git clone https://github.com/jsha/minica.git
go build

As Go is a compiled language, this will build an executable in your current directory called Minica. You can then create your SSL certs quickly and easily:

./minica --domains splunkfwd.idelta.co.uk
./minica --domains splunkidx.idelta.co.uk
./minica --domains splunkweb.idelta.co.uk

Using openssl we can inspect the certificate generated and see that we have a certificate with a common name: CN=splunkweb.idelta.co.uk

openssl x509 -in splunkweb.idelta.co.uk/cert.pem -noout -text
        Version: 3 (0x2)
        Serial Number: 5042391479290351192 (0x45fa2c55ad8ece58)
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN=minica root ca 209eeb
            Not Before: Apr 15 21:29:20 2021 GMT
            Not After : May 15 21:29:20 2023 GMT
        Subject: CN=splunkweb.idelta.co.uk

Note that the expiry date of the certificate is, by default, 2 years and 1 month. You can easily change this by altering the following line in the “sign” method. Edit the source code file main.go and find the following line within the sign method. The three numbers refer to years, months and days:

NotAfter: time.Now().AddDate(2, 0, 30),

Remember that Go is a compiled language so you’ll need to “go build” after making the changes. In the example below we changed the line above to set a “NotAfter” date of 5 years from now.

openssl x509 -in splunkfwdlong.idelta.co.uk/cert.pem -noout -text
        Version: 3 (0x2)
        Serial Number: 3138546151390316011 (0x2b8e5b1890ca05eb)
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN=minica root ca 209eeb
            Not Before: Apr 15 21:37:05 2021 GMT
            Not After : Apr 15 21:37:05 2026 GMT
        Subject: CN=splunkfwdlong.idelta.co.uk

We can now quickly set up splunkweb to use the new private key and cert as follows:

cd /opt/splunk/etc/apps/etc/apps
mkdir idelta_splunkwebssl
cd idelta_splunkwebssl/
cp <path-to-certs>/key.pem .
cp <path-to-certs>/cert.pem .
mkdir local
cd local
vi web.conf

enableSplunkWebSSL = true
privKeyPath = etc/apps/idelta_splunkwebssl/key.pem
serverCert = etc/apps/idelta_splunkwebssl/cert.pem

Restart splunk and load the splunk web login page. As this new certificate has been signed by a CA not known or trusted by your browser, you’ll get a warning. View the certificate and you’ll see that the new cert is in use.

Further information on configuring Splunk to use SSL can be found in the Splunk docs.

Certificate expiration and how you handle renewals is an important part of keeping your production Splunk installation operational and secure. Distributing new certificates via a Deployment Server is an efficient way of updating certificates before they expire but you want to make sure your process is right before sending out the new app. Using the methods detailed above you can setup a certificate due to expire soon and then generate another replacement certificate with the same common name to replace it. Work through the process in your dev environment to prove your steps before going live.

The minica GitHub page is here: https://github.com/jsha/minica

For 2021 we’ve committed to posting a new Splunk tip every week!

If you want to keep up to date on tips like the one above then sign up below:

Subscribe to our newsletter to receive regular updates from iDelta, including news and updates, information on upcoming events, and Splunk tips and tricks from our team of experts. You can also find us on Twitter and LinkedIn.


* indicates required
Posted by:Stuart Robertson

Stuart Robertson is the Consulting Director at iDelta. He is one of the initial founders of iDelta and has worked there since formation in 2001. Stuart holds various certifications in Core Splunk and ITSI. Stuart also holds a Bsc(Hons) in Computing Science from the University of Glasgow.