docker, ammonite, servers, scripting
etl, training, development, open-source
- allows non-privileged users to run their scripts in a sandboxed environment (Docker containers)
- scripts can be organized as parents and siblings (in a directed acyclic graph)
- “child” scripts read their input from their “parents” ouptut
- architectured as one “master” server (Kabuto project) and several “computing nodes” (Ammonite project)
- computing nodes communicate over HTTP and message queues
- fallback system designed to recover from dead computing nodes and restart the job it was doing
- master server can be horizontally load-balanced to handle a large number of computing nodes
- input is always read-only, this way a malicious user cannot damage existing files
