Data Engineer 資料工程師
CurrentConstruction and Management of Data Pipelines1. Routinely retrieved and updated data from external platforms via APIs, incorporating retry mechanisms and ensuring data accuracy.2. Maintained data quality by investigating anomalies during updates and effectively resolving false alarms.3. Integrated cloud and on-premise pipelines to ensure data users consistently access accurate information.Cloud Migration of Data Pipelines1. Successfully transitioned the on-premises pipeline to Azure Synapse Analytics and restructured it to eliminate unnecessary dependencies.2. Refactored the on-premises stored procedure for migration, achieving a 6-10x increase in execution speed.3. Implemented a daily data validation mechanism to identify and record any