com.mozilla.secops.parser

Class ParserMultiDoFn

java.lang.Object

org.apache.beam.sdk.transforms.DoFn<org.apache.beam.sdk.values.KV<String,String>,org.apache.beam.sdk.values.KV<String,Event>>

com.mozilla.secops.parser.ParserMultiDoFn

All Implemented Interfaces:

Serializable, org.apache.beam.sdk.transforms.display.HasDisplayData

public class ParserMultiDoFn
extends org.apache.beam.sdk.transforms.DoFn<org.apache.beam.sdk.values.KV<String,String>,org.apache.beam.sdk.values.KV<String,Event>>

Process an incoming raw event feed using multiple parser configurations

Similar to ParserDoFn, but can be used to process an incoming raw event feed that is keyed using an identifier. An arbitrary number of parser configurations and filters can be installed in the class associated with an identifier.

When the DoFn processes an incoming raw string, it will utilize the configuration and filters associated with the identifier corresponding to the key of the input string.

The output will be the parsed event, keyed with the same identifier as the raw string.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.beam.sdk.transforms.DoFn

org.apache.beam.sdk.transforms.DoFn.AlwaysFetched, org.apache.beam.sdk.transforms.DoFn.BoundedPerElement, org.apache.beam.sdk.transforms.DoFn.BundleFinalizer, org.apache.beam.sdk.transforms.DoFn.Element, org.apache.beam.sdk.transforms.DoFn.FieldAccess, org.apache.beam.sdk.transforms.DoFn.FinishBundle, org.apache.beam.sdk.transforms.DoFn.FinishBundleContext, org.apache.beam.sdk.transforms.DoFn.GetInitialRestriction, org.apache.beam.sdk.transforms.DoFn.GetInitialWatermarkEstimatorState, org.apache.beam.sdk.transforms.DoFn.GetRestrictionCoder, org.apache.beam.sdk.transforms.DoFn.GetSize, org.apache.beam.sdk.transforms.DoFn.GetWatermarkEstimatorStateCoder, org.apache.beam.sdk.transforms.DoFn.Key, org.apache.beam.sdk.transforms.DoFn.MultiOutputReceiver, org.apache.beam.sdk.transforms.DoFn.NewTracker, org.apache.beam.sdk.transforms.DoFn.NewWatermarkEstimator, org.apache.beam.sdk.transforms.DoFn.OnTimer, org.apache.beam.sdk.transforms.DoFn.OnTimerContext, org.apache.beam.sdk.transforms.DoFn.OnTimerFamily, org.apache.beam.sdk.transforms.DoFn.OnWindowExpiration, org.apache.beam.sdk.transforms.DoFn.OnWindowExpirationContext, org.apache.beam.sdk.transforms.DoFn.OutputReceiver<T>, org.apache.beam.sdk.transforms.DoFn.ProcessContext, org.apache.beam.sdk.transforms.DoFn.ProcessContinuation, org.apache.beam.sdk.transforms.DoFn.ProcessElement, org.apache.beam.sdk.transforms.DoFn.RequiresStableInput, org.apache.beam.sdk.transforms.DoFn.RequiresTimeSortedInput, org.apache.beam.sdk.transforms.DoFn.Restriction, org.apache.beam.sdk.transforms.DoFn.Setup, org.apache.beam.sdk.transforms.DoFn.SideInput, org.apache.beam.sdk.transforms.DoFn.SplitRestriction, org.apache.beam.sdk.transforms.DoFn.StartBundle, org.apache.beam.sdk.transforms.DoFn.StartBundleContext, org.apache.beam.sdk.transforms.DoFn.StateId, org.apache.beam.sdk.transforms.DoFn.Teardown, org.apache.beam.sdk.transforms.DoFn.TimerFamily, org.apache.beam.sdk.transforms.DoFn.TimerId, org.apache.beam.sdk.transforms.DoFn.Timestamp, org.apache.beam.sdk.transforms.DoFn.TruncateRestriction, org.apache.beam.sdk.transforms.DoFn.UnboundedPerElement, org.apache.beam.sdk.transforms.DoFn.WatermarkEstimatorState, org.apache.beam.sdk.transforms.DoFn.WindowedContext

Constructor Summary

Constructors

Constructor and Description
ParserMultiDoFn() Create new ParserMultiDoFn

Method Summary

Modifier and Type	Method and Description
void	addParser(String name, ParserCfg cfg, EventFilter inlineFilter) Add a new parser configuration and filter for the specified key name
void	processElement(org.apache.beam.sdk.transforms.DoFn.ProcessContext c)
void	setup()

Methods inherited from class org.apache.beam.sdk.transforms.DoFn

getAllowedTimestampSkew, getInputTypeDescriptor, getOutputTypeDescriptor, populateDisplayData, prepareForProcessing

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ParserMultiDoFn

public ParserMultiDoFn()

Create new ParserMultiDoFn

Method Detail

addParser

public void addParser(String name,
                      ParserCfg cfg,
                      EventFilter inlineFilter)

Add a new parser configuration and filter for the specified key name

Parameters:

name - Identifier name

cfg - Parser configuration

inlineFilter - Inline event filter, null for none

setup

@DoFn.Setup
public void setup()

processElement

@DoFn.ProcessElement
public void processElement(org.apache.beam.sdk.transforms.DoFn.ProcessContext c)