Data Engineer
Current• Created CDC pipeline using Apache Kafka for archiving online databases.• Optimized Airflow Operators to improve speed and fail-safety of ETL processes.• Rebuilt DWH from scratch, resulting in 50% less processing time and better data quality.• Developed migration pipelines to drastically decrease data transfer latency and ensure resilience to schema alterations.