I’ve attended the last four Splunk .conf and I can honestly say that they look nothing alike!
I was quite new in the Spunk world when I traveled for my first conference in Orlando and I absolutely loved it! The reason is that I discovered numerous use cases and ways to ingest the data that I was not yet aware of. Mind you, I wouldn’t have thought I would see somebody using Splunk to retrieve and analyze data on his Harley Davidson! It definitely was a fun session and I still can remember it precisely for this reason.
This year was also interesting but in a different manner: new features, capabilities and connectivities were added to Splunk thanks to some acquisitions by Splunk to widen and sharpen their beloved toolset. As my role, knowledge and projects have evolved over the years, I have encountered some challenges that I’m always keen on solving and keep an eye on the ways to do so.
This year, I discovered Syslog Connector for Splunk, which was created by two splunkers (Ryan Faircloth and Mark Bonsack) in their spare time. I want to write about it because it relates so much to some of the problems we can encounter on an every day basis. The session was called : “Take Control of Port 514!: Taming the Syslog Beast” and needless to say I was already intrigued.
Syslog is a protocol that has been around for about three decades and is actively used in most companies nowadays. When ingesting syslog data into Splunk, the challenges can be multiple:
- One sourcetype being used for all type of data
- Syslog configuration itself
- Mainly uses UDP by default, which is not reliable
The main idea behind this session was to ask people to stop sending directly to the splunk Indexers but to use the Splunk Connect for Syslog (SC4S) instead. The question answered here is “How do I easily ingest syslog data, at scale, while removing the requirement of up-front design work and syslog-fu?”.
Why is it interesting?
The exciting part of the speech was the capability to have a sourcetype given to each different event without having to use several TAs (Technical Add-ons) or Transformations that would look at each event and decide whether they belong to them or not. You can now understand that on a small instance it is not important, however on a large scale it becomes resource intensive for the Indexers.
The other selling point is the use of HEC (HTTP Event Collector) which is something that has been widely adopted by our customers. We can therefore see that with this SC4S project, we can improve on a technical point of view our current setup and align with the customer’s strategic decisions for their infrastructure.
What’s more, it comes with some self monitoring metrics. You will be able to see in real time how your input rate is charting and scale your infrastructure based on that info.
Finally, this solution comes with containers using Red Hat Linux UBI (Universal Base Image) that are considered secure and can be easily trusted by your Security team, as well as pre-configured parsing for most event types. This allows you to scale up and down quickly.
Did I mention it’s even Open Source?
Here is what it looks like:
If that sounds like something you would like to use, feel free to follow this link and listen to this amazing session: