Routing Windows events

Routing events into two or more different indexes is one of those questions that has been asked many times on Splunk Answers and various blogs. This blog post will provide a comprehensive overview of how Windows events can be routed into two different indexes following 4 simple steps.

Use Case

You are working on a project that requires you to split Windows events into two different indexes. For example, you may want to send EventCode=4624 into index1 and all of the remaining events codes into index2.

Solution

First, we need to create our indexes, for example:

[all_ad_data]
homePath = $SPLUNK_DB/all_ad_data/db
coldPath = $SPLUNK_DB/all_ad_data/colddb
thawedPath = $SPLUNK_DB/all_ad_data/thaweddb
maxTotalDataSizeMB = 512000
enableDataIntegrityControl = 0
enableTsidxReduction = 0

[selected_events]
homePath = $SPLUNK_DB/selected_events/db
coldPath = $SPLUNK_DB/selected_events/colddb
thawedPath = $SPLUNK_DB/selected_events/thaweddb
maxTotalDataSizeMB = 512000
enableDataIntegrityControl = 0
enableTsidxReduction = 0

Second, create props.conf, for example:

[ad_security]
SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n]+)\d{2}\/\d{2}\/\d{4}\s[\d+:]+\s[[:upper:]]+
TIME_FORMAT=%d/%m/%Y %H:%M:%S %p
TIME_PREFIX=^
TRUNCATE=20000
TRANSFORMS-newindex = selected_events

If you need to route events to more that one index, then you would specify any other indexes in TRANSFORMS-newindex in a comma separated list

Let’s break down the props.conf and transforms reference:

[<spec>]
TRANSFORMS-<class> = <unique_stanza_name>

Where:

• <spec> is your sourcetype name. The same as you specified in your inputs.conf
• <class> is any unique identifier that you want to give to your transform
• <unique_stanza_name> is the name of the stanza you will create in transforms.conf

Third, let’s create our transforms.conf that uses the same <unique_stanza_name>  as you specified in your props.conf:

[selected_events]
REGEX = .*EventCode=4624
FORMAT = selected_events
DEST_KEY =_MetaData:Index

Let’s break the above configuration down:

REGEX = Your regex that identifies a common attribute for the events that can be used to differentiate them. In this case we want to route events based on the values in the EventCode field.
FORMAT = specifies the alternative index name that the events will routed to.
DEST_KEY = key value that specifies where Splunk software stores the expanded FORMAT results in accordance with the REGEX match

https://docs.splunk.com/Documentation/Splunk/7.3.1/Admin/Transformsconf

Forth, create inputs.conf, for example:

[monitor:///home/splunk/logs/ad_test/ad_security.log1]
index=all_ad_data
sourcetype=ad_security
disabled=0

Notice that in our inputs.conf we only have one monitoring stanza which references the index that will contain all of the remaining event codes, i.e, ones that do not need to be routed.
Please also note that, based on your Splunk set up, you may also need to configure serverclass.

End Result

You should now receive data from one source into two different indexes, one only containing selected event codes and the other containing the remaining event codes.