Data Engineer
Current- Built and optimized multiple data pipelines through Apache Airflow; proactively monitoring pipeline performance, identifying and resolving bottlenecks to ensure optimal throughput and reliability.
- Engaged extensively with GCP services such as Identity & Access Management (IAM), Google Cloud Storage (GCS), BigQuery, Dataform, Dataflow, Dataplex, Vertex AI, Cloud Billing, and additional services to enhance data.
- Authored and polished Terraform templates for GCP to establish development, staging and production environments, enabling consistent, reliable and efficient infrastructure management.
- Developed and improved comprehensive documentation for ETL processes and architecture ensuring clarity for team members and stakeholders, which facilitated onboarding and knowledge transfer.
- Partnered closely with data scientists, analysts and other stakeholders to identify data needs and provide tailored, effective solutions.