Storm, Business Intelligence

ETL framework for financial and scientific reporting
Engie Global Markets, Energy sector
  • Replace an existing workflow based on Excel in order to:
    • Federate team work around a same referential of data.
    • Streamline and automate the day to day process.
    • Measure and monitor with continuous integration the solution performance with respect to historical data.
  • Maintain information about power plants in Europe
    • Data is stored in Postgresql in ~40 tables.
    • Loading and update of data is done mainly from or to Excel files.
  • Generate all input files necessary to run the price model
    • Plugable architecture to handle data transformation (based on pandas, numpy and some scientific Python libraries).
    • Declarative definition of generated files.
    • Collect live information form tens dozens of sources (either through web scraping or from coprporate web-services).