Calculating Session Duration in Logstash Using Ruby – Source: socprime.com

When processing event streams with Logstash, you may encounter a need to calculate the duration of a session — the difference between its start and end times. This is a common scenario when analyzing time-sensitive data.
Logstash provides the flexibility to perform such operations by embedding custom Ruby code into your pipeline configuration. Here’s how you can achieve this.

Scenario

Presumably, our event data includes two fields:

• startTime: The timestamp when the session began.
• endTime: The timestamp when the session ended.

To calculate the session duration and add it as a new field, you can use the Ruby filter plugin.

Solution

Add the following Ruby code to your Logstash pipeline configuration:

ruby {     code => '         if event.get("startTime") and event.get("endTime")             duration = event.get("endTime").to_i - event.get("startTime").to_i             event.set("eventduration", duration)         end     ' }

Explanation of the Code

1. Input Validation: the if condition checks whether both startTime and endTime fields exist in the event. This prevents errors when these fields are missing.
2. Duration Calculation:
   • event.get("startTime").to_i: Converts the startTime value to an integer (usually a Unix timestamp).
   • event.get("endTime").to_i: Converts the endTime value to an integer.
   • The difference (endTime – startTime) calculates the duration in seconds.
3. Setting the Output: the result is added as a new field, eventduration, to the event.

Integrating into Your Pipeline

Here’s how the Ruby filter might fit into a basic pipeline configuration:

input {     file {         path => "https://socprime.com/path/to/your/logfile.log"         start_position => "beginning"     } }  filter {     ruby {         code => '             if event.get("startTime") and event.get("endTime")                 duration = event.get("endTime").to_i - event.get("startTime").to_i                 event.set("eventduration", duration)             end         '     } }  output {     elasticsearch {         hosts => ["http://localhost:9200"]         index => "sessions-index"     }     stdout { codec => json } }

Usage Notes

• Timestamps Format: Ensure startTime and endTime are in a numeric format (e.g., Unix epoch). If they are in a different format, you may need to preprocess them using the date filter.
• Field Names: Replace startTime and endTime with the actual field names in your data, if they differ.
• Error Handling: Optionally, add logging or a fallback mechanism for cases where the time fields are invalid or missing.

Benefits

• Efficiency: The Ruby filter enables quick, inline calculations without the need for external scripts.
• Customizable: You can extend the Ruby code for more complex operations, such as handling edge cases or logging errors.

By embedding this Ruby code in your Logstash configuration, you can automate session duration calculations and enrich your event data for further analysis in Elasticsearch.

Was this article helpful?

Like and share it with your peers.