Data And Analytics Engineer
Current• Architected and maintained 80+ data pipelines leveraging Python, Bash, Airflow, and Docker, ensuring resilient data integration with more than 30 systems and 25 Tb within the Hadoop and Oracle ecosystem.• Collaborated with Data Analysts to identify and implement data model improvements, resulting in a 20% decrease in tables required for processing, which streamlined data management and improved overall efficiency.• Designed and implemented infrastructure recommendations to transition to a DevOps/DataOps approach using git, GitLab pipelines and Ansible, enhancing workflow efficiency and streamlining development processes for the Data Engineering team.• Led the design and implementation of 20% of the data model, driving enhancements and optimizations leveraging dbt.• Carry out full system data migration with 25Tb of data with over 90 schemas and thousands of tables.