Package | Description |
---|---|
com.mozilla.secops |
General utility classes and transforms for secops-beam
|
com.mozilla.secops.amo |
AMO analysis pipeline
|
com.mozilla.secops.authprofile |
Pipeline for authentication source profiling and alerting
|
com.mozilla.secops.awsbehavior |
Pipeline for monitoring AWS Cloudtrail events
|
com.mozilla.secops.customs |
Customs FxA analysis pipeline
|
com.mozilla.secops.gatekeeper |
Pipeline for AWS Guardduty and GCP ETD analysis
|
com.mozilla.secops.httprequest |
HTTP request threshold and error rate monitoring
|
com.mozilla.secops.httprequest.heuristics | |
com.mozilla.secops.input |
Pipeline input
|
com.mozilla.secops.parser |
Log parsing, processing, and enrichment
|
com.mozilla.secops.pioneer |
Pioneer analysis pipeline
|
com.mozilla.secops.postprocessing |
Pipeline for further processing of and correlation between alerts
|
Modifier and Type | Method and Description |
---|---|
static org.apache.beam.sdk.transforms.DoFn<Event,Event> |
CidrUtil.excludeNormalizedSourceAddresses(int flags,
String path)
Returns a DoFn that filters any events that have a normalized source address field that matches
the specified criteria.
|
static org.apache.beam.sdk.transforms.DoFn<Event,Event> |
CidrUtil.excludeNormalizedSourceAddresses(int flags,
String path)
Returns a DoFn that filters any events that have a normalized source address field that matches
the specified criteria.
|
Modifier and Type | Method and Description |
---|---|
org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<String,Boolean>> |
DetectNat.UserAgentBased.expand(org.apache.beam.sdk.values.PCollection<Event> events) |
static org.apache.beam.sdk.values.PCollectionView<Map<String,Boolean>> |
DetectNat.getView(org.apache.beam.sdk.values.PCollection<Event> events,
String knownGatewaysPath)
Execute nat detection transforms returning a
PCollectionView suitable for use as a side
input, currently only User Agent Based |
Modifier and Type | Method and Description |
---|---|
org.apache.beam.sdk.values.PCollection<Alert> |
AddonMultiIpLogin.expand(org.apache.beam.sdk.values.PCollection<Event> col) |
org.apache.beam.sdk.values.PCollection<Alert> |
ReportRestriction.expand(org.apache.beam.sdk.values.PCollection<Event> col) |
org.apache.beam.sdk.values.PCollection<Alert> |
AddonMultiSubmit.expand(org.apache.beam.sdk.values.PCollection<Event> col) |
org.apache.beam.sdk.values.PCollection<Alert> |
AddonCloudSubmission.expand(org.apache.beam.sdk.values.PCollection<Event> col) |
org.apache.beam.sdk.values.PCollection<Alert> |
AddonMatcher.expand(org.apache.beam.sdk.values.PCollection<Event> col) |
org.apache.beam.sdk.values.PCollection<Alert> |
FxaAccountAbuseAlias.expand(org.apache.beam.sdk.values.PCollection<Event> col) |
org.apache.beam.sdk.values.PCollection<Alert> |
AddonMultiMatch.expand(org.apache.beam.sdk.values.PCollection<Event> col) |
org.apache.beam.sdk.values.PCollection<Alert> |
FxaAccountAbuseNewVersion.expand(org.apache.beam.sdk.values.PCollection<Event> col) |
Modifier and Type | Method and Description |
---|---|
org.apache.beam.sdk.values.PCollection<Event> |
AwsAssumeRoleCorrelator.expand(org.apache.beam.sdk.values.PCollection<Event> input) |
org.apache.beam.sdk.values.PCollection<Event> |
AuthProfile.Parse.expand(org.apache.beam.sdk.values.PCollection<String> col) |
Modifier and Type | Method and Description |
---|---|
static Alert |
AuthProfile.createBaseAlert(Event e,
String contactEmail,
String docLink)
Create a base authprofile
Alert using information from the event |
static void |
AuthProfile.insightsEnrichAlert(Alert a,
Event e)
Add minfraud insights data into alert metadata
|
Modifier and Type | Method and Description |
---|---|
org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<String,Alert>> |
CritObjectAnalyze.expand(org.apache.beam.sdk.values.PCollection<Event> input) |
org.apache.beam.sdk.values.PCollection<Event> |
AwsAssumeRoleCorrelator.expand(org.apache.beam.sdk.values.PCollection<Event> input) |
Modifier and Type | Method and Description |
---|---|
org.apache.beam.sdk.values.PCollection<Event> |
AwsBehavior.ParseAndWindow.expand(org.apache.beam.sdk.values.PCollection<String> col) |
Modifier and Type | Method and Description |
---|---|
org.apache.beam.sdk.values.PCollection<Alert> |
AwsBehavior.Matcher.expand(org.apache.beam.sdk.values.PCollection<Event> col) |
org.apache.beam.sdk.values.PCollection<Alert> |
AwsBehavior.Matchers.expand(org.apache.beam.sdk.values.PCollection<Event> col) |
Modifier and Type | Field and Description |
---|---|
static org.apache.beam.sdk.values.TupleTag<Event> |
CustomsPreFilter.TAG_FXA_AUTH_EVENTS
Tuple tag used for FxA auth events
|
static org.apache.beam.sdk.values.TupleTag<Event> |
CustomsPreFilter.TAG_FXA_CONTENT_EVENTS |
static org.apache.beam.sdk.values.TupleTag<Event> |
CustomsPreFilter.TAG_RELAY_EVENTS
Tuple tag used for private relay events
|
Modifier and Type | Method and Description |
---|---|
org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<String,Event>> |
CustomsWindow.FixedTenMinutes.expand(org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<String,Event>> input) |
ArrayList<Event> |
CustomsFeatures.getEvents()
Get event list
|
ArrayList<Event> |
CustomsFeatures.getEventsOfType(FxaAuth.EventSummary t)
Get all events from event list of a certain type
|
Modifier and Type | Method and Description |
---|---|
void |
CustomsFeatures.addEvent(Event e)
Add a single event to the event list
|
CustomsFeatures |
CustomsFeaturesCombiner.CustomsFeaturesCombineFn.addInput(CustomsFeatures col,
Event input) |
static FxaAuth |
CustomsUtil.authGetData(Event e)
Extract FxA event internal data
|
static String |
CustomsUtil.authGetEmail(Event e)
Extract FxA event email address
|
static FxaAuth.EventSummary |
CustomsUtil.authGetEventSummary(Event e)
Extract FxA event summary
|
static String |
CustomsUtil.authGetPath(Event e)
Extract FxA event path
|
static FxaAuth |
CustomsUtil.authGetPayload(Event e)
Extract FxA event payload
|
static String |
CustomsUtil.authGetService(Event e)
Extract FxA event service value
|
static String |
CustomsUtil.authGetSourceAddress(Event e)
Extract FxA event source address
|
static Double |
CustomsUtil.authGetSourceAddressLatitude(Event e)
Extract FxA event source address latitude
|
static Double |
CustomsUtil.authGetSourceAddressLongitude(Event e)
Extract FxA event source address longitude
|
static Integer |
CustomsUtil.authGetStatus(Event e)
Extract FxA event status code
|
static String |
CustomsUtil.authGetUid(Event e)
Extract FxA event UID
|
static String |
CustomsUtil.authGetUserAgent(Event e)
Extract FxA agent
|
Modifier and Type | Method and Description |
---|---|
org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<String,Boolean>> |
ContentServerVarianceDetector.PresenceBased.expand(org.apache.beam.sdk.values.PCollection<Event> events) |
org.apache.beam.sdk.values.PCollection<Alert> |
CustomsVelocity.expand(org.apache.beam.sdk.values.PCollection<Event> col) |
org.apache.beam.sdk.values.PCollection<Alert> |
Customs.CustomsSummary.expand(org.apache.beam.sdk.values.PCollection<Event> col) |
org.apache.beam.sdk.values.PCollection<Alert> |
PrivateRelayForward.expand(org.apache.beam.sdk.values.PCollection<Event> col) |
org.apache.beam.sdk.values.PCollection<Alert> |
CustomsStatusComparator.expand(org.apache.beam.sdk.values.PCollection<Event> col) |
org.apache.beam.sdk.values.PCollection<Alert> |
CustomsActivityForMonitoredAccounts.expand(org.apache.beam.sdk.values.PCollection<Event> col) |
org.apache.beam.sdk.values.PCollection<Alert> |
CustomsLoginFailureForAtRiskAccount.expand(org.apache.beam.sdk.values.PCollection<Event> col) |
org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<String,Event>> |
CustomsWindow.FixedTenMinutes.expand(org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<String,Event>> input) |
org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<String,CustomsFeatures>> |
CustomsFeaturesCombiner.expand(org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<String,Event>> input) |
static org.apache.beam.sdk.values.PCollectionView<Map<String,Boolean>> |
ContentServerVarianceDetector.getView(org.apache.beam.sdk.values.PCollection<Event> events)
Execute transform returning a
PCollectionView of ips accessing content server
resources, that can be used as a side input. |
void |
CustomsFeatures.setEvents(ArrayList<Event> events)
Set event list
|
Modifier and Type | Method and Description |
---|---|
org.apache.beam.sdk.values.PCollection<Event> |
GuardDutyTransforms.ExtractFindings.expand(org.apache.beam.sdk.values.PCollection<Event> input) |
org.apache.beam.sdk.values.PCollection<Event> |
ETDTransforms.ExtractFindings.expand(org.apache.beam.sdk.values.PCollection<Event> input) |
org.apache.beam.sdk.values.PCollection<Event> |
GatekeeperParser.Parse.expand(org.apache.beam.sdk.values.PCollection<String> rawInputStrings) |
Modifier and Type | Method and Description |
---|---|
org.apache.beam.sdk.values.PCollection<Event> |
GuardDutyTransforms.ExtractFindings.expand(org.apache.beam.sdk.values.PCollection<Event> input) |
org.apache.beam.sdk.values.PCollection<Alert> |
GuardDutyTransforms.GenerateGDAlerts.expand(org.apache.beam.sdk.values.PCollection<Event> input) |
org.apache.beam.sdk.values.PCollection<Event> |
ETDTransforms.ExtractFindings.expand(org.apache.beam.sdk.values.PCollection<Event> input) |
org.apache.beam.sdk.values.PCollection<Alert> |
ETDTransforms.GenerateETDAlerts.expand(org.apache.beam.sdk.values.PCollection<Event> input) |
Modifier and Type | Method and Description |
---|---|
org.apache.beam.sdk.values.PCollection<Event> |
HTTPRequest.WindowForFixed.expand(org.apache.beam.sdk.values.PCollection<Event> input) |
org.apache.beam.sdk.values.PCollection<Event> |
HTTPRequestElementFilter.expand(org.apache.beam.sdk.values.PCollection<Event> col) |
Modifier and Type | Method and Description |
---|---|
Boolean |
HTTPRequest.Has4xxRequestStatus.apply(Event event) |
Modifier and Type | Method and Description |
---|---|
org.apache.beam.sdk.values.PCollection<Event> |
HTTPRequest.WindowForFixed.expand(org.apache.beam.sdk.values.PCollection<Event> input) |
org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<String,ArrayList<String>>> |
HTTPRequest.KeyAndWindowForSessionsFireEarly.expand(org.apache.beam.sdk.values.PCollection<Event> input) |
org.apache.beam.sdk.values.PCollection<Event> |
HTTPRequestElementFilter.expand(org.apache.beam.sdk.values.PCollection<Event> col) |
Modifier and Type | Method and Description |
---|---|
org.apache.beam.sdk.values.PCollection<Alert> |
ThresholdAnalysis.expand(org.apache.beam.sdk.values.PCollection<Event> col) |
org.apache.beam.sdk.values.PCollection<Alert> |
StatusCodeRateAnalysis.expand(org.apache.beam.sdk.values.PCollection<Event> input) |
org.apache.beam.sdk.values.PCollection<Alert> |
EndpointSequenceAbuse.expand(org.apache.beam.sdk.values.PCollection<Event> input) |
org.apache.beam.sdk.values.PCollection<Alert> |
ErrorRateAnalysis.expand(org.apache.beam.sdk.values.PCollection<Event> input) |
org.apache.beam.sdk.values.PCollection<Alert> |
HardLimitAnalysis.expand(org.apache.beam.sdk.values.PCollection<Event> input) |
org.apache.beam.sdk.values.PCollection<Alert> |
UserAgentBlocklistAnalysis.expand(org.apache.beam.sdk.values.PCollection<Event> input) |
Modifier and Type | Method and Description |
---|---|
org.apache.beam.sdk.values.PCollection<Event> |
Input.SimplexReader.expand(org.apache.beam.sdk.values.PBegin begin) |
org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<String,Event>> |
Input.MultiplexReader.expand(org.apache.beam.sdk.values.PBegin begin) |
org.apache.beam.sdk.values.PCollection<Event> |
InputElement.expandElement(org.apache.beam.sdk.values.PBegin begin,
String project)
Expand configured input types into a resulting collection of parsed events
|
org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PBegin,org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<String,Event>>> |
Input.multiplexRead()
Return a transform that will ingest data, and emit parsed events in multiplex mode
|
org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PBegin,org.apache.beam.sdk.values.PCollection<Event>> |
Input.simplexRead()
Return a transform that will ingest data, and emit parsed events in simplex mode
|
Modifier and Type | Method and Description |
---|---|
static Event |
Event.fromJSON(String input)
Convert a JSON string into an
Event |
Event |
Parser.parse(String input)
Parse an event
|
Modifier and Type | Method and Description |
---|---|
static org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<Event>,org.apache.beam.sdk.values.PCollection<Event>> |
EventFilter.getTransform(EventFilter filter)
Get composite transform to apply filter to event stream
|
static org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<Event>,org.apache.beam.sdk.values.PCollection<Event>> |
EventFilter.getTransform(EventFilter filter)
Get composite transform to apply filter to event stream
|
static Iterable<Event> |
Event.jsonToIterable(String input)
Utility function to convert a JSON string into an iterable list of events
|
org.apache.beam.sdk.values.KV<String,Event> |
KeyedEvent.toKV()
Convert KeyedEvent to
KV |
Modifier and Type | Method and Description |
---|---|
Boolean |
EventFilter.matches(Event e)
Test if event matches filter
|
Boolean |
EventFilterPayloadInterface.matches(Event e)
Should return true if the filter matches the supplied event
|
Boolean |
EventFilterPayloadOr.matches(Event e)
Return true if payload criteria matches
|
Boolean |
EventFilterPayload.matches(Event e)
Return true if payload criteria matches
|
Boolean |
EventFilterRule.matches(Event e)
Test if event matches rule
|
static org.joda.time.DateTime |
Parser.parseAndCorrectSyslogTs(String in,
Event e)
Parse syslog timestamp date time string and return a
DateTime object using Parser.parseSyslogTs(String) , and then correct the year if the parsed timestamp is further than
three days from the event timestamp. |
Modifier and Type | Method and Description |
---|---|
static org.joda.time.DateTime |
Parser.getLatestTimestamp(Iterable<Event> events)
Given an interable of events, return the latest timestamp
|
static String |
Event.iterableToJson(Iterable<Event> input)
Utility function to convert an iterable list of events into a JSON string
|
Constructor and Description |
---|
Alert(String input,
Event e,
com.mozilla.secops.parser.ParserState state)
Construct parser object.
|
AmoDocker(String input,
Event e,
com.mozilla.secops.parser.ParserState state)
Construct parser object.
|
ApacheCombined(String input,
Event e,
com.mozilla.secops.parser.ParserState state)
Construct parser object.
|
Auth0(String input,
Event e,
com.mozilla.secops.parser.ParserState state)
Construct parser object.
|
BmoAudit(String input,
Event e,
com.mozilla.secops.parser.ParserState state)
Construct parser object.
|
CfgTick(String input,
Event e,
com.mozilla.secops.parser.ParserState state)
Construct parser object.
|
Cloudtrail(String input,
Event e,
com.mozilla.secops.parser.ParserState state)
Construct parser object.
|
Duopull(String input,
Event e,
com.mozilla.secops.parser.ParserState state)
Construct parser object.
|
ETDBeta(String input,
Event e,
com.mozilla.secops.parser.ParserState s)
Construct parser object.
|
FxaAuth(String input,
Event e,
com.mozilla.secops.parser.ParserState state)
Construct parser object.
|
FxaContent(String input,
Event e,
com.mozilla.secops.parser.ParserState state)
Construct parser object.
|
GcpAudit(String input,
Event e,
com.mozilla.secops.parser.ParserState state)
Construct parser object.
|
GcpVpcFlow(String input,
Event e,
com.mozilla.secops.parser.ParserState state)
Construct parser object.
|
GLB(String input,
Event e,
com.mozilla.secops.parser.ParserState state)
Construct parser object.
|
GuardDuty(String input,
Event e,
com.mozilla.secops.parser.ParserState s)
Construct parser object.
|
IPrepdLog(String input,
Event e,
com.mozilla.secops.parser.ParserState s)
Construct parser object.
|
KeyedEvent(String key,
Event event)
Initialize new KeyedEvent
|
Nginx(String input,
Event e,
com.mozilla.secops.parser.ParserState state)
Construct parser object.
|
OpenSSH(String input,
Event e,
com.mozilla.secops.parser.ParserState state)
Construct parser object.
|
PayloadBase(String input,
Event e,
com.mozilla.secops.parser.ParserState state)
Construct parser object.
|
Phabricator(String input,
Event e,
com.mozilla.secops.parser.ParserState state)
Construct parser object.
|
PrivateRelay(String input,
Event e,
com.mozilla.secops.parser.ParserState state)
Construct parser object.
|
Raw(String input,
Event e,
com.mozilla.secops.parser.ParserState state)
Construct parser object.
|
Taskcluster(String input,
Event e,
com.mozilla.secops.parser.ParserState state)
Construct parser object.
|
Modifier and Type | Method and Description |
---|---|
org.apache.beam.sdk.values.PCollection<Alert> |
Pioneer.PioneerExfiltration.expand(org.apache.beam.sdk.values.PCollection<Event> col) |
Modifier and Type | Method and Description |
---|---|
org.apache.beam.sdk.values.PCollection<Event> |
PostProcessing.Parse.expand(org.apache.beam.sdk.values.PCollection<String> col) |
Copyright © 2022. All rights reserved.