Data Engineer
Current- Architected an end-to-end data processing pipeline that consolidated multi-source datasets and tables from SQL Server, boosting analytics efficiency. - Build and designed best practices to support CI/CD for data ingestion and data pipeline workflows using Azure Data Factory and DevOps. - Developed an automated ETL process using Python and Scala notebooks on Databricks. - Worked closely with third parties and client to resolve issues through JIRA. - Created a notebook with metrics to guarantee data quality in machine learning algorithm. - Visualized data on dashboards with Power BI.