Big Data Engineer, Manager
Current- Developed real-time data pipelines using PySpark Streaming and Kafka SQL, improving data availability by up to 24 hours.
- Collaborated with stakeholders to define data pipeline requirements for personalization and recommendation use cases.
- Enhanced business agility by automating real-time data extraction, transformation, and loading (ETL).