com.mozilla.secops

Class SourceCorrelation.SourceCorrelator

java.lang.Object

org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<SourceCorrelation.SourceData>,org.apache.beam.sdk.values.PCollection<Alert>>

com.mozilla.secops.SourceCorrelation.SourceCorrelator

All Implemented Interfaces:

DocumentingTransform, Serializable, org.apache.beam.sdk.transforms.display.HasDisplayData

Enclosing class:

SourceCorrelation

public static class SourceCorrelation.SourceCorrelator
extends org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<SourceCorrelation.SourceData>,org.apache.beam.sdk.values.PCollection<Alert>>
implements DocumentingTransform

Transform for source address alert and ingestion correlation

This transform can be used to correlate source address information from an ingestion event stream with source address information that is present in alerts generated by the pipeline.

Currently this transform is limited to ISP based correlation. Alerts are grouped by ISP based on distinct source address, and are compared with source address and ISP information grouped in a similar way from the ingestion event stream.

If at least a certain number of addresses are seen associated with a given ISP, and the percentage of these addresses that generated an alert in a fixed window exceeds the configured value, an alert is generated.

Currently uses a hardcoded fixed window value of 6 hours.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
SourceCorrelator(HTTPRequestToggles toggles) Initialize new SourceCorrelator

Method Summary

Modifier and Type	Method and Description
org.apache.beam.sdk.values.PCollection<Alert>	expand(org.apache.beam.sdk.values.PCollection<SourceCorrelation.SourceData> input)
String	getTransformDoc() Get documentation string from transform based on it's current configuration

Methods inherited from class org.apache.beam.sdk.transforms.PTransform

compose, compose, getAdditionalInputs, getDefaultOutputCoder, getName, getResourceHints, populateDisplayData, setResourceHints, toString, validate

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

SourceCorrelator

public SourceCorrelator(HTTPRequestToggles toggles)

Initialize new SourceCorrelator

Parameters:

toggles - HTTPRequestToggles

Method Detail

getTransformDoc

public String getTransformDoc()

Get documentation string from transform based on it's current configuration

Specified by:

getTransformDoc in interface DocumentingTransform

Returns:

String

expand

public org.apache.beam.sdk.values.PCollection<Alert> expand(org.apache.beam.sdk.values.PCollection<SourceCorrelation.SourceData> input)

Specified by:

expand in class org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<SourceCorrelation.SourceData>,org.apache.beam.sdk.values.PCollection<Alert>>