BI ETL/ELT Framework (bi_etl)
Python based Extract Transform Load (ETL) or Extract Load Transform (ELT) framework geared towards Business Intelligence (BI) dimensional databases in particular. The goal of the project is to create reusable objects with typical technical transformations used in loading dimension tables.
Sources supported:
- Database tables
- Database SQL Queries
- Delimited text files
- Excel files
- W3C web logs
Targets supported:
- Delimited text files
- Excel files
- Database tables
- Works with any database supported by SQL Alchemy
- Simple inserts
- Update else Insert (upsert)
- Versioned Update else Insert (upsert SCD Type 2)
- Sourced based versioned Update else Insert
- ALL of the above support
- Memory/disk caching for performance
- Bulk loading the results