com.mozilla.secops.httprequest

Class HTTPRequest

java.lang.Object

com.mozilla.secops.httprequest.HTTPRequest

All Implemented Interfaces:

Serializable

public class HTTPRequest
extends Object
implements Serializable

HTTPRequest describes and implements a Beam pipeline for analysis of HTTP requests using log data.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	HTTPRequest.Has4xxRequestStatus Function to be used with filter transform in order to include only client errors
static interface	HTTPRequest.HTTPRequestOptions Runtime options for HTTPRequest pipeline.
static class	HTTPRequest.KeyAndWindowForSessionsFireEarly Key requests for session analysis and window into sessions
static class	HTTPRequest.WindowForFixed Window events into fixed one minute windows

Constructor Summary

Constructors

Constructor and Description
HTTPRequest()

Method Summary

Modifier and Type	Method and Description
static void	addToggleCacheEntry(String name, HTTPRequestToggles entry) Add an entry to the HTTPRequest toggle cache
static String	buildConfigurationTick(HTTPRequest.HTTPRequestOptions options, HTTPRequestToggles toggles) Build a configuration tick for HTTPRequest given pipeline options and configuration toggles
static org.apache.beam.sdk.values.PCollection<Alert>	expandInputMap(org.apache.beam.sdk.Pipeline p, org.apache.beam.sdk.values.PCollectionTuple input, HTTPRequest.HTTPRequestOptions options) Expand the input collection tuple, executing analysis transforms for each element
static Input	getInput(org.apache.beam.sdk.Pipeline p, HTTPRequest.HTTPRequestOptions options) Given HTTPRequest pipeline options, return a configured Input class
static void	main(String[] args) Entry point for Beam pipeline.
static org.apache.beam.sdk.values.PCollectionTuple	readInput(org.apache.beam.sdk.Pipeline p, Input input, HTTPRequest.HTTPRequestOptions options) Read from a configured Input object, returning a PCollectionTuple of events

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HTTPRequest

public HTTPRequest()

Method Detail

addToggleCacheEntry

public static void addToggleCacheEntry(String name,
                                       HTTPRequestToggles entry)

Add an entry to the HTTPRequest toggle cache

For use in tests, should not be called under normal use.

Parameters:

name - Cache key name

entry - Cache entry

buildConfigurationTick

public static String buildConfigurationTick(HTTPRequest.HTTPRequestOptions options,
                                            HTTPRequestToggles toggles)
                                     throws IOException

Build a configuration tick for HTTPRequest given pipeline options and configuration toggles

Parameters:

options - Pipeline options

toggles - Analysis toggles

Returns:

String

Throws:

IOException - IOException

getInput

public static Input getInput(org.apache.beam.sdk.Pipeline p,
                             HTTPRequest.HTTPRequestOptions options)
                      throws IOException

Given HTTPRequest pipeline options, return a configured Input class

The returned input object will be configured based on the HTTPRequest pipeline options, and will have all applicable filters assigned.

Parameters:

p - Pipeline

options - Pipeline options

Returns:

Configured Input object

Throws:

IOException - IOException

readInput

public static org.apache.beam.sdk.values.PCollectionTuple readInput(org.apache.beam.sdk.Pipeline p,
                                                                    Input input,
                                                                    HTTPRequest.HTTPRequestOptions options)

Read from a configured Input object, returning a PCollectionTuple of events

The PCollectionTuple that is returned contains PCollections tagged by resource indicator. Each PCollection contains only events and uses the resource indicator as the TupleTag id.

Parameters:

p - Pipeline

input - Configured Input object

options - Pipeline options

Returns:

PCollectionTuple with each event collection tagged by resource indicator

expandInputMap

public static org.apache.beam.sdk.values.PCollection<Alert> expandInputMap(org.apache.beam.sdk.Pipeline p,
                                                                           org.apache.beam.sdk.values.PCollectionTuple input,
                                                                           HTTPRequest.HTTPRequestOptions options)

Expand the input collection tuple, executing analysis transforms for each element

Parameters:

p - Pipeline

input - PCollectionTuple using resource name as tag id

options - Pipeline options

Returns:

Flattened alerts in the global window

main

public static void main(String[] args)
                 throws Exception

Entry point for Beam pipeline.

Parameters:

args - Runtime arguments.

Throws:

Exception - Exception