Kirby, event-driven, distributed, data collection framework

kafka, etl, celery, monitoring
big-data, etl, open-source
  • similar to Apache Airflow
  • uses Kafka to store event data, PostgreSQL for jobs
  • built to be highly resilient
  • fully tested and documented
  • realtime monitoring of data flowing through the system