Data Engineer
Currentโข Streamlined ETL processes using GCP Cloud Dataflow, improving data integrity by 25%, increasing throughput by 30%, and reducing conversion errors by 15% through robust data validation mechanisms.โข Automated workflows with Cloud Composer (Apache Airflow), minimizing manual intervention and enhancing the reliability of data pipelines across GCP services.โข Leveraged Google Cloud Storage (GCS), BigQuery, and Compute Engine to architect scalable data solutions, optimizing efficiency in data warehousing, storage, and virtual machine management.โข Collaborated with cross-functional teams to gather, document, and translate business requirements into technical specifications, ensuring alignment with GCP-based data solutions.โข Led optimization efforts for BigQuery, Cloud SQL, and Cloud Spanner databases, achieving a 30% reduction in query execution time by implementing advanced indexing, partitioning, and tuning techniques.โข Developed Python-based automation for seamless data ingestion into GCS and Cloud SQL, ensuring data accuracy and integrity throughout the pipeline.โข Implemented metadata management and governance practices in Dataproc, establishing data lineage, quality checks, and audit trails to ensure compliance and data governance.โข Designed cloud-native data warehouse solutions using BigQuery, optimizing scalability and performance across large datasets, reducing data processing time by 20%.โข Created and optimized Looker dashboards and reports, leveraging SQL and LookML to extract insights and support business stakeholders in data-driven decision-making.โข Created automated dashboards using Looker, and Power BI, providing real-time insights that empower business stakeholders to make informed decisions and drive strategic initiatives.โข Introduced CI/CD practices for data pipelines in Google Cloud Build and Cloud Functions, streamlining the development, testing, and deployment of cloud-based solutions.