docker, ammonite, servers, scripting
etl, training, development, open-source
- allows non-privileged users to run their scripts in a sandboxed environment (Docker containers)
- scripts can be organized as parents and siblings (in a directed acyclic graph)
- “child” scripts read their input from their “parents” ouptut
- architectured as one “master” server (Kabuto project) and several “computing nodes” (Ammonite project)
- computing nodes communicate over HTTP and message queues
- fallback system designed to recover from dead computing nodes and restart the job it was doing
- master server can be horizontally load-balanced to handle a large number of computing nodes
- input is always read-only, this way a malicious user cannot damage existing files
![Kabuto, grid computing 1 rabbitmq logo strap](https://www.adimian.com/wp-content/uploads/2021/10/rabbitmq_logo_strap.png)
![Kabuto, grid computing 2 docker](https://www.adimian.com/wp-content/uploads/2021/10/docker-1024x914.png)
![Kabuto, grid computing 3 python](https://www.adimian.com/wp-content/uploads/2021/10/python.png)