Data Engineer
CurrentContributed to the development of data ingestion pipelines from APIs and multi-cloud in a Data Lake, using Python, Pandas, Airflow, Spark, DBT, and S3 for ingestion into a Redshift data warehouse. My responsibilities included data integration and transformation from various sources, workflow orchestration with Airflow, and pipeline performance optimization, ensuring data quality and consistency through continuous validation and monitoring processes.I worked on the optimization and support of critical data infrastructures, applying DataOps practices to ensure the continuity of complex pipelines. I was responsible for automation and observability using Grafana and Prometheus, ensuring SLAs were met and maintaining operational resilience.I was also responsible for cost optimization with FinOps in AWS and GCP environments, adapting architectures to maximize efficiency and minimize expenses. Developed dashboards in Power BI to provide cost visibility, supporting decision-making and ensuring data governance and compliance.