com.mozilla.secops.parser

Class Parser

java.lang.Object

com.mozilla.secops.parser.Parser

public class Parser
extends Object

Event parser

Parser can be used to parse incoming events and generate Event objects.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Parser.EventTooOldException Indicates the extracted event timestamp was too old

Field Summary

Fields

Modifier and Type	Field and Description
static String[]	onlyStackdriverTypes If a Stackdriver log message contains any strings present in this array in the @type field, we assume no other encapsulation is present in the parser
static String	SYSLOG_TS_RE

Constructor Summary

Constructors

Constructor and Description
Parser() Create new parser instance with default configuration
Parser(ParserCfg cfg) Create new parser instance with specified configuration

Method Summary

Modifier and Type	Method and Description
String	applyProxyXFFAddressSelector(String input, Boolean proxyPipeline) Applies proxy xff selector
String	applyXffAddressSelector(String input) Apply any configured XFF address selector to the specified input string
static <T,U> HashMap<T,U>	convertJsonToMap(String input) Utility function to convert a JSON string into the desired map type
static <T,U> HashMap<T,U>	convertJsonToMap(String input, com.fasterxml.jackson.databind.ObjectMapper mapper) Utility function to convert a JSON string into the desired map type
com.maxmind.geoip2.model.CityResponse	geoIp(String ip) Resolve GeoIP information from IP address string
com.maxmind.geoip2.model.IspResponse	geoIpIsp(String ip) Resolve GeoIP ISP information from IP address string
IdentityManager	getIdentityManager() Get any configured identity manager from the parser
static org.joda.time.DateTime	getLatestTimestamp(Iterable<Event> events) Given an interable of events, return the latest timestamp
Event	parse(String input) Parse an event
static org.joda.time.DateTime	parseAndCorrectSyslogTs(String in, Event e) Parse syslog timestamp date time string and return a DateTime object using parseSyslogTs(String), and then correct the year if the parsed timestamp is further than three days from the event timestamp.
static org.joda.time.DateTime	parseISO8601(String in) Parse an ISO8601 date string and return a DateTime object.
static org.joda.time.DateTime	parseSyslogTs(String in) Parse syslog timestamp date time string and return a DateTime object.
static String[]	parseXForwardedFor(String in) Process the value of an X-Forwarded-For header, returning an array of each address in the header or null if invalid
void	setIdentityManager(IdentityManager idmanager) Set an identity manager in the parser that can be used for lookups
Boolean	shouldUseXff() Returns true if using an XFF header is enabled in the parser

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

SYSLOG_TS_RE

public static final String SYSLOG_TS_RE

See Also:

Constant Field Values

onlyStackdriverTypes

public static final String[] onlyStackdriverTypes

If a Stackdriver log message contains any strings present in this array in the @type field, we assume no other encapsulation is present in the parser

Constructor Detail

Parser

public Parser(ParserCfg cfg)

Create new parser instance with specified configuration

Parameters:

cfg - ParserCfg

Parser

public Parser()

Create new parser instance with default configuration

Method Detail

getLatestTimestamp

public static org.joda.time.DateTime getLatestTimestamp(Iterable<Event> events)

Given an interable of events, return the latest timestamp

Parameters:

events - Event list

Returns:

Latest timestamp in event set

parseSyslogTs

public static org.joda.time.DateTime parseSyslogTs(String in)

Parse syslog timestamp date time string and return a DateTime object.

Parameters:

in - Input string

Returns:

Parsed DateTime, null if string could not be parsed

parseAndCorrectSyslogTs

public static org.joda.time.DateTime parseAndCorrectSyslogTs(String in,
                                                             Event e)

Parse syslog timestamp date time string and return a DateTime object using parseSyslogTs(String), and then correct the year if the parsed timestamp is further than three days from the event timestamp.

Parameters:

in - Input string

e - Event

Returns:

Parsed DateTime, null if string could not be parsed

parseISO8601

public static org.joda.time.DateTime parseISO8601(String in)

Parse an ISO8601 date string and return a DateTime object.

Parameters:

in - Input string

Returns:

Parsed DateTime, null if string could not be parsed

applyXffAddressSelector

public String applyXffAddressSelector(String input)
                               throws IllegalArgumentException

Apply any configured XFF address selector to the specified input string

If no XFF address selector has been configured in the parser configuration, and the input contains multiple XFF style addresses, the last address is returned.

Parameters:

input - Input string

Returns:

Results of address selector application

Throws:

IllegalArgumentException

shouldUseXff

public Boolean shouldUseXff()

Returns true if using an XFF header is enabled in the parser

Returns:

Whether the parser is configured to use an XFF header

applyProxyXFFAddressSelector

public String applyProxyXFFAddressSelector(String input,
                                           Boolean proxyPipeline)

Applies proxy xff selector

Parameters:

input - input to apply selector to

proxyPipeline - whether this is marked as through the proxy

Returns:

the result of the proxy selector

convertJsonToMap

public static <T,U> HashMap<T,U> convertJsonToMap(String input)

Utility function to convert a JSON string into the desired map type

Type Parameters:

T - T

U - U

Parameters:

input - Input JSON

Returns:

HashMap

convertJsonToMap

public static <T,U> HashMap<T,U> convertJsonToMap(String input,
                                                  com.fasterxml.jackson.databind.ObjectMapper mapper)

Utility function to convert a JSON string into the desired map type

Type Parameters:

T - T

U - U

Parameters:

input - Input JSON

mapper - ObjectMapper

Returns:

HashMap

parseXForwardedFor

public static String[] parseXForwardedFor(String in)

Process the value of an X-Forwarded-For header, returning an array of each address in the header or null if invalid

Parameters:

in - Input string

Returns:

Array of addresses, beginning from the left-most, or null on failure

geoIp

public com.maxmind.geoip2.model.CityResponse geoIp(String ip)

Resolve GeoIP information from IP address string

GeoIP resolution must be enabled in the parser, or this function will always return null.

Parameters:

ip - IP address string

Returns:

MaxmindDB CityResponse, or null if lookup fails

geoIpIsp

public com.maxmind.geoip2.model.IspResponse geoIpIsp(String ip)

Resolve GeoIP ISP information from IP address string

GeoIP resolution must be enabled in the parser with the ISP database, or this function will always return null.

Parameters:

ip - IP address string

Returns:

MaxmindDB IspResponse, or null if lookup fails

setIdentityManager

public void setIdentityManager(IdentityManager idmanager)

Set an identity manager in the parser that can be used for lookups

Parameters:

idmanager - Initialized IdentityManager

getIdentityManager

public IdentityManager getIdentityManager()

Get any configured identity manager from the parser

Returns:

IdentityManager or null if no manager has been set in the parser

parse

public Event parse(String input)
            throws Parser.EventTooOldException

Parse an event

Parameters:

input - Input string

Returns:

Event or null if the event should be ignored

Throws:

Parser.EventTooOldException - EventTooOldException