Mozilla Security Client Error Rate

For events matching the message_matcher, this analysis plugin identifies the number of events seen for a given identification field (e.g., the IP address in an nginx log) that have resulted in an HTTP client error code (e.g., >= 400, < 500).

Any clients that have exceeded error_threshold errors in a given ticker interval will be added to a list that is processed during the timer event.

If send_iprepd is true, violation messages will be generated for the iprepd output plugin using the specified violation_type. If send_iprepd is false, TSV output will be created containing the violation list for the ticker interval.

If enable_metrics is true, the module will submit metrics events for collection by the metrics output sandbox. Ensure timer_event_inject_limit is set appropriately, as if enabled timer_event will submit up to 2 messages (the violation notice, and the metric event).

1. Sample Configuration

filename = "moz_security_http_error_rate.lua"
message_matcher = "Logger == 'input.nginx'"
ticker_interval = 60
preserve_data = false

id_field = "Fields[remote_addr]" -- field to use as the identifier
-- id_field_capture = ",? *([^,]+)$",  -- optional e.g. extract the last entry in a comma delimited list
code_field = "Fields[code]" -- field to extract HTTP status code from

-- list_max_size = 500 -- optional, defaults to 500 (maximum number of clients that can be flagged per tick)
-- send_iprepd = false -- optional, if true plugin will generate iprepd violation messages
-- violation_type = "fxa:client_error_rate" -- required in violations mode, iprepd violation type
error_threshold = 50 -- clients generating over error_threshold client errors will be tracked

-- cms_epsilon = 1 / 10000 -- optional CMS value for epsilon
-- cms_delta = 0.0001 -- optional CMS value for delta

-- enable_metrics = false -- optional, if true enable secmetrics submission

source code: moz_security_http_error_rate.lua