public class ParserCfg extends Object implements Serializable
Parser
Constructor and Description |
---|
ParserCfg()
Construct default parser configuration
|
Modifier and Type | Method and Description |
---|---|
static ParserCfg |
fromInputOptions(InputOptions options)
Create a parser configuration from pipeline
InputOptions |
Boolean |
getDeferGeoIpResolution()
Get defer GeoIP resolution setting
|
Boolean |
getDisableCloudwatchStrip()
Get disable Cloudwatch strip flag
|
Boolean |
getDisableMozlogStrip()
Get disable Mozlog strip flag
|
String |
getIdentityManagerPath()
Get IdentityManager json file path
|
String |
getMaxmindCityDbPath()
Get Maxmind City database path
|
String |
getMaxmindIspDbPath()
Get Maxmind ISP database path
|
Integer |
getMaxTimestampDifference()
Get maximum allowable timestamp difference
|
String |
getParserFastMatcher() |
String[] |
getStackdriverLabelFilters()
Get Stackdriver label filters
|
String |
getStackdriverProjectFilter()
Get Stackdriver project filter
|
Boolean |
getUseEventTimestamp()
Get event timestamp emission setting
|
Boolean |
getUseProxyXff()
Get whether to use the proxy header to select ip from XFF
|
Boolean |
getUseXffAsRemote()
Get Use Xff Header as Remote
|
ArrayList<String> |
getXffAddressSelector()
Get any configured XFF address selectors
|
CidrUtil |
getXffAddressSelectorAsCidrUtil()
Return any configured XFF address selectors as a
CidrUtil object. |
void |
setDeferGeoIpResolution(Boolean deferGeoIpResolution)
Set defer GeoIP resolution
|
void |
setDisableCloudwatchStrip(boolean disableCloudwatchStrip)
Set disable Cloudwatch strip
|
void |
setDisableMozlogStrip(boolean disableMozlogStrip)
Set disable Mozlog strip
|
void |
setIdentityManagerPath(String path)
Set IdentityManager json file path
|
void |
setMaxmindCityDbPath(String path)
Set Maxmind City database path
|
void |
setMaxmindIspDbPath(String path)
Set Maxmind ISP database path
|
void |
setMaxTimestampDifference(Integer maxTimestampDifference)
Set maximum allowable timestamp difference
|
void |
setParserFastMatcher(String fastMatcher)
Set parser fast matcher
|
void |
setStackdriverLabelFilters(String[] stackdriverLabelFilters)
Set Stackdriver label filters
|
void |
setStackdriverProjectFilter(String stackdriverProjectFilter)
Set Stackdriver project filter
|
void |
setUseEventTimestamp(Boolean useEventTimestamp)
Set event timestamp emission setting
|
void |
setUseProxyXff(Boolean useProxyXff)
Set enable proxy xff
|
void |
setUseXffAsRemote(Boolean xffAsRemote)
Parse the X-Forwarded-For header instead of the remote addr
|
void |
setXffAddressSelector(ArrayList<String> subnets)
Set XFF address selectors
|
public static ParserCfg fromInputOptions(InputOptions options)
InputOptions
options
- Input optionspublic void setXffAddressSelector(ArrayList<String> subnets)
The subnets parameter should be an ArrayList containing CIDR subnets that will be used as hints for selecting a real client IP address in the event parsers see an X-Forwarded-For style address list.
If any address in the log entry address list matches a subnet in the configured selector list, the address directly to the left will be used as the real client IP address.
If this value is not set, the rightmost address will always be used as the actual client IP address.
This option is intended to behave in a similar manner to the nginx realip module, https://nginx.org/en/docs/http/ngx_http_realip_module.html.
subnets
- Arraylist containing subnetspublic ArrayList<String> getXffAddressSelector()
ArrayList
of subnets, or null if unsetpublic CidrUtil getXffAddressSelectorAsCidrUtil()
CidrUtil
object.public String getMaxmindCityDbPath()
public void setMaxmindCityDbPath(String path)
path
- Pathpublic String getMaxmindIspDbPath()
public void setMaxmindIspDbPath(String path)
path
- Pathpublic String getIdentityManagerPath()
public void setIdentityManagerPath(String path)
path
- Pathpublic String getParserFastMatcher()
public void setParserFastMatcher(String fastMatcher)
If a fast matcher is set in the parser, an input string is immediately tested to see if it contains the supplied substring. If not, the event is dropped prior to performing the bulk of the parsing/filtering operations.
Note that this is intended to reduce pressure on the parser and should not be treated as a filter, as there are certain cases (such as with configuration tick events) that messages that do not match the fast matcher will still be returned.
fastMatcher
- Matcher substringpublic Boolean getUseEventTimestamp()
public void setUseEventTimestamp(Boolean useEventTimestamp)
This option is only applicable when the parser is being used within ParserDoFn
.
If true, events will be output in the pipeline using the timestamp within the event rather than the default timestamp that would be assigned.
useEventTimestamp
- Booleanpublic String[] getStackdriverLabelFilters()
public void setStackdriverLabelFilters(String[] stackdriverLabelFilters)
This option is only applicable when the parser is being used within ParserDoFn
.
If set, events that do not match the label specification will be dropped.
stackdriverLabelFilters
- Array of key:value labels that must matchpublic String getStackdriverProjectFilter()
public void setStackdriverProjectFilter(String stackdriverProjectFilter)
This option is only applicable when the parser is being used within ParserDoFn
If set, events that do not have the specified Stackdriver project value will be dropped.
stackdriverProjectFilter
- Project valuepublic void setDisableCloudwatchStrip(boolean disableCloudwatchStrip)
If it is known ahead of time the parser will never have to strip cloudwatch encapsulation off an event, this flag can be enabled which will increase parser performance.
disableCloudwatchStrip
- Booleanpublic Boolean getDisableCloudwatchStrip()
public void setDisableMozlogStrip(boolean disableMozlogStrip)
If it is known ahead of time the parser will never have to strip mozlog encapsulation off an event, this flag can be enabled which will increase parser performance.
disableMozlogStrip
- Booleanpublic Boolean getDisableMozlogStrip()
public void setMaxTimestampDifference(Integer maxTimestampDifference)
If set, events which are parsed will have the timestamp included with the event compared against the current time. If the difference exceeds the specified value in seconds, the event will be dropped.
By default, this value is not set. Note that not all payload parsers will extract and set an event timestamp. In cases where this does not happen, the event timestamp will just default to the time the event was parsed (current time).
maxTimestampDifference
- Max timestamp difference in secondspublic Integer getMaxTimestampDifference()
public void setDeferGeoIpResolution(Boolean deferGeoIpResolution)
If set, GeoIP resolution on events will not actually occur until a GeoIP related field is read. Otherwise, it will occur immediately when a source address field is set if GeoIP is enabled.
deferGeoIpResolution
- Booleanpublic Boolean getDeferGeoIpResolution()
public void setUseProxyXff(Boolean useProxyXff)
If set, preprocesses the remote addr chain based on proxy header presence.
useProxyXff
- Booleanpublic Boolean getUseProxyXff()
public void setUseXffAsRemote(Boolean xffAsRemote)
If set, parse XFF header, if present, in supported message types and enable usage of the XFF header. This is explicitly enabled to support inconsistent log formats where multiple IPs may also be used in remote addr fields.
xffAsRemote
- Booleanpublic Boolean getUseXffAsRemote()
Copyright © 2022. All rights reserved.