Senior Consultant, Machine Learning Engineer
Woonsocket, Ri, Us
- Built an in-house framework to standardize building, testing, deploying data pipelines using different platforms, for example, Azure data factory, snowflake, kubeflow, docker etc. - Optimized the existing data pipelines’ performance by sorting out the dependency between steps. Helped the team to scale the pipeline vertically instead of horizontally to save money for the team.- Build tools to connect teradata server and Azure Gen2 data lake. It makes the data migration from teradata to Gen2 5x faster than before.- Built tools to monitor and alert the usage of the Azure Kubernetes Cluster to prevent abnormal abuse activities from other teams.