Heka Sandbox Interface

1. Overview

This document describes the 1.0 release of the Heka sandbox API built on the Generic Sandbox Interface. The 1.0 release is a refined implementation of its predecessor which was developed in Heka. The goal is to decouple it from Go and make it easily embeddable in any language. The Go version of Heka has been deprecated and replaced by Hindsight.

2. Sandbox API Changes from the Go Heka Sandbox

There are a few intentional changes between tho original Heka sandbox and this version.

2.1. Changes

2.1.1. Input Sandbox

1. inject_message accepts a numeric or string checkpoint as the second argument
2. process_message receives the checkpoint value as the first argument (if it was provided by inject_message).

2.1.2. Output Sandbox

1. process_message receives a sequence_id as the first argument (if it was provided by update_checkpoint). Extended return codes have been added to support skipping, retrying, batching, and asynchronous output.

2.1.3. Analysis/Output Sandbox

1. read_message
   • returns nil for optional header fields if they don't exist instead of an empty string or zero
   • added a framed parameter to retrieve the raw message with stream framing
   • added a size parameter to retrieve size of the raw message without having to copy it down
2. timer_event has a second parameter shutdown that is set to true when the sandbox is exiting.

2.2. Additions

2.2.1. Input Sandbox

1. create_stream_reader function was added.
2. is_running function was added.

2.2.2. Output Sandbox

1. update_checkpoint was added for batch and asynchronous processing.
2. create_message_matcher function was added.

2.3. Removals

1. The write_message API was removed; messages are immutable and this API broke that rule.
2. The read_next_field API was removed; instead the raw message should be decoded and the Lua table iterated.

2.4. Notes

1. The read_config API in unchanged but now has access to the entire sandbox configuration.