Mozilla Server Log Data S3 Input Sandbox Bootstrapper

This sandbox:

1. retrieves a list of files from S3 matching the service log name and the date range
2. divides the list among the specified number of partitions
3. generates the cfgs to dynamically start a new reader for each partition; the specified input_plugin should already be installed in run/input
4. exits when the bootstrapping is complete

1. Sample Configuration

filename            = "moz_serverlog_s3_bootstrap.lua"
instruction_limit   = 0
ticker_interval     = 0

input_plugin        = "heka_s3.lua"
input_plugin_cfgs   = {} -- table of sandbox specific config options
tmp_dir             = "/mnt/work/tmp"
s3_bucket           = "heka-logs"
s3_prefix           = "shared"
start_date          = "2016-04-01"
end_date            = "2016-04-07"
service             = "loop-app"
partitions          = 1

source code: moz_serverlog_s3_bootstrap.lua