Azure Data Engineer
Current• Design and development, unit testing, integration, deployment packaging and checkout, scheduling, of various components in Informatica Power Center, Azure Data Factory and Azure Data Bricks through several SDLCs for implementing ETL/ELT processes for cloud based very large data warehouse and ODS integrated with data for subject areas from disparate data sources and processes supporting downstream data marts.• Performed analysis of ETL/ELT applications and database systems, data structure analysis, data profiling for application design and defects resolution.• Converted legacy ETL processes into Azure synapse compatible architecture using regular expression driven metadata conversion such as ODBC/ JDBC reads and OLAP writers and replacements such as bulk merges, deletes with OLAP model.• Performed process enhancements such as capacity improvement, tuning/optimization and performance improvement of the Azure Hyperscale based ETL/Upsert processes.• Performed extensive metadata gathering and analysis of database and metadata repositories for understanding data lineage, data flow, process design, process statistics and identifying storage elements and access to data using JSON, XML, AVRO.• Partnered with DBA/SA to optimize database tables/indexes, partitioning, parallelization and in implementing database/client configurations to support pipeline partitioning and multi-threaded processing.