Senior Data Engineer
Current- As a Senior Data Engineer (B2C) I turn business needs from internal leaders into data pipelines used across teams within Glassdoor to support our internal data infrastructure. Using tools such as Airflow, I have.
- Leading a proof-of-concept initiative with Confluent Kafka to reduce latency and cost (by 10s of thousands of dollars) of consumer metrics from daily batch processing to near real-time, using Kafka to transport data.
- Developed ETL pipeline to assist in migration of A/B testing framework to Optimizely, using the Optimizely API to download batch A/B testing events, S3 sensors to check when data has landed, and a custom Hive operator.