The Splunk Processing Language (SPL) provides powerful statistical functions which enable you to calculate summary statistics on the results of a search or the events retrieved from an index. In this blog post we will look at eventstats and streamstats, which are variations of the stats command in splunk. From Splunk documentation, “The eventstats command calculates statistics on all search results and adds the aggregation inline to each event for which it is relevant. The streamstats command calculates statistics for each event at the time the event is seen, in a streaming manner.”
Eventstats and streamstats are centralized streaming commands, meaning they apply a transformation to each event returned by a search. Like the stats command, a number of statistical functions are available such as avg() , count() , distinct_count() , median() , perc<int>() , stdev() , sum() , sumsq() , etc. These commands are processed on the search head only.
To illustrate the how these commands work, I will use the commands to calculate the total bytes for web requests and the average bytes for web requests logged by a Cisco Web Security Appliance. For simplicity, I will limit the result set to 5 events. In the Code Listing 1, “cs_username”, bytes_in, bytes_out are fields associated with the cisco-wsa-squid eventtype and bytes is a calculated field. The stats command is used with a sum and avg functions to find the total web request bytes (total_req_bytes) and the average bytes (avg_req_bytes) of web requests by cs_username, shown in Figure 1. With the reduce results set, there is one request per cs_username, therefore total_req_bytes = avg_req_bytes.
Code Listing 1: Sum and Average of Web Request Bytes by cs_username
eventtype=cisco-wsa-squid action=”allow”
|head 5
| eval bytes=bytes_in + bytes_out
| stats sum(bytes) as total_req_bytes, avg(bytes) as avg_req_bytes by cs_username
Figure 1: Code Listing 1 Statistics Table
The eventstats command finds events that contain the field that you want to aggregate and creates a new field in every event and places the aggregation in that field. In Code Listing 2, the eventstats command is used to calculate the sum and average of all the web request for all the returned events. As shown in Figure 2, for all events the sum of total_req_bytes = 48865 and the average of total_req_bytes = 12216.25
Code Listing 2: Using Splunk Eventstats for Sum and Average of Web Request Bytes
eventtype=cisco-wsa-squid action=”allow” | head 5
| eval bytes=bytes_in + bytes_out
| stats sum(bytes) as total_req_bytes by cs_username
| eventstats sum(total_req_bytes) as a_total_req_bytes, avg(total_req_bytes) as b_avg_req_bytes
Figure 2: Code Listing 2 Statistics Table
Splunk streamstats command calculates a “running total” of the selected field and creates a new field in every event and places the aggregation in that field. In Code Listing 3, the streamstats command is used to calculate the running total and running average of web requests as events are seen. As shown in Figure 3, as events are seen the sum of total_req_bytes and the average of total_req_bytes results are updated. When the last result is seen the sum of total_req_bytes and the average of total_req_bytes are the same value as the result from eventstats.
Code Listing 3: Using Splunk Streamstats for Sum and Average of Web Request Bytes
eventtype=cisco-wsa-squid action=”allow” | head 5
| eval bytes=bytes_in + bytes_out
| stats sum(bytes) as total_req_bytes by cs_username
| streamstats sum(total_req_bytes) as a_run_total_req_bytes, b_run_avg(total_req_bytes) as b_avg_req_bytes
Figure 3: Code Listing 3 Statistics Table
In this blog post, I have described the eventstats and streamstats commands and demonstrated their usage. I hope you will find this information to be a useful aid in leveraging these commands in your searches.
Happy Splunking!
