Data Engineer
Current- Developed a data trash collection utility using IICS and a bash script that runs daily to delete files and sources exceeding the threshold days, while also notifying stakeholders of any folder modifications via email.- Transformed the stored procedures of the relational data vault into optimized IICS Taskflows, enhancing performance and providing data lineage for ingested data.- Improved existing RDV Taskflows by integrating an audit process, which captures execution details during taskflow runs.- Collaborated with the data modeling team to analyze and categorize relational data vault tables into their classifications.- Worked on ingesting various data sources into BigQuery.