Searching the itsi_summary index provides a convenient way to access ITSI KPI values in a normal Splunk events search. Alongside this index however is the ITSI Summary Metrics (itsi_summary_metrics) index which as the name suggests is a metrics index and can therefore provide an even more efficient method of accessing the KPI data. The metrics summary index was introduced with ITSI version 4.6.0.
Preview the metrics
Using the “mpreview” command we can take a look at the fields within the metrics:
|mpreview index=itsi_summary_metrics filter="metric_name=alert_value"
We can see the itsi_kpi_id and itsi_service_id are both available.
Reporting Using the ITSI Summary Metrics Index
Depending on the what the KPI represents we can use “mstats” and the sum or avg aggregators to quickly calculate values over a wide time range, likely a much longer time range than available in your splunk data events index.
The following search calculates the average (mean), max and standard deviation of a response time KPI:
mstats avg(alert_value) as avg_alert_value max(alert_value) as max_alert_value stdev(alert_value) as stdev_alert_value max(alert_level) as max_alert_level where index=itsi_summary_metrics AND itsi_kpi_id=5a13eeaa1a7a9fff317204be
Timecharts using ITSI Summary Metrics Index
The example below shows how the metrics index can be used to quickly create a timechart using the ITSI Summary Metrics Index. We can quickly split the data into time blocks by simply adding a “span=” clause to the end of the search.
| mstats avg(alert_value) as avg_alert_value max(alert_value) as max_alert_value stdev(alert_value) as stdev_alert_value max(alert_level) as max_alert_level where index=itsi_summary_metrics AND itsi_kpi_id=5a13eeaa1a7a9fff317204be is_service_max_severity_event=0 span=4h
As you are simply searching a metrics index, that happens to store ITSI KPI data, you don’t need to be on the ITSI search head in order to run these searches. You can use these methods on Splunk Core search heads and build the results into Splunk Core dashboards. See this previous post on how that could be useful.
Splunk docs provides a reference for the ITSI Metrics Index.
For 2021 we’ve committed to posting a new Splunk tip every week!
If you want to keep up to date on tips like the one above then sign up below:
Subscribe to our newsletter to receive regular updates from iDelta, including news and updates, information on upcoming events, and Splunk tips and tricks from our team of experts. You can also find us on Twitter and LinkedIn.