Senior Data Engineer
Current• Established a PostgreSQL data warehouse with automated ETL processes via Fivetran• Implemented DBT (Data Build Tool) pipelines for all data warehouse tasks, ensuring efficient and reliable data transformations• Deployed Metabase open source instance and connected the data warehouse to facilitate self-service analytics and automate BI reporting• Advanced infrastructure as code (IaC) adoption using Terraform, facilitating scalable and consistent infrastructure provisioning and management• Developed and maintained various robust, fault-tolerant web scraping solutions with mechanisms to handle site changes, bot detection, dynamic content loading, and rate limits