Data Engineer
Current- Project: Real-time Data Ingestion and Processing Pipeline on Uber ride-hailing platform
- Collaborate frequently with senior team members to implement data transformation tasks such as filtering, and aggregating. Specify windowing strategies for handling streaming data using Apache Beam for stream processing.
- Assists in configuring Google Cloud Platform (GCP) services, including setting up Pub/Sub for event streaming involving creating topics to publish messages and subscriptions to receive messages for processing in the.
- Conduct data quality checks, monitor pipeline logs, and assist in troubleshooting data pipeline issues to ensure smooth data flow. List of Data Engineering concept I am comfortable with: - Filtering- Aggregating.