Data Engineer Intern
Scottsdale, Az, Us
Documented current processes of BI data pipeline with AsciiDoc to understand complex data transformations, change data capture procedures (CDC), ETL pipelines, and data architecture for 500 million+ records across 50+ processes to provide a high-level understanding of the data pipeline.Developed and optimized scripts emulating the functionality of the documented processes (Full, Incremental, CDC) currently being performed by SAP Data Services (BODS) using Transactional SQL (TSQL), SISS, Git SSMS, Azure Data Studio, Data Warehousing, Data Modeling, and Data Architecture to discontinue use of the service and internalize its functionality into the organization.Tested, validated, and deployed developed scripts and processes in 4 environments DEV, QA, UA and PROD, sequentially through Data Validation scripts, SSIS packages to ensure smooth integration, deployment and usage of production data across multiple departments.