• Successfully migrated data from traditional database systems to Azure, enhancing scalability and efficiency.• Generated ER diagrams, architectural design diagrams, process flow documents, data dictionaries, and mapping documents, facilitating project understanding and collaboration.• Wrote complex transformation scripts to populate dimension tables, enhancing data processing efficiency.• Engineered end-to-end metadata-driven pipelines, ensuring atomicity and re usability for streaming, batch, and event-driven processing. (Using ADF, Data bricks, Power Bi etc.)• Create Views, stored procedures and write complex SQL queries as per the requirements.• Designed and implemented metadata models for centralized custom logging and data quality framework, enhancing data integrity across sources. (Using Data bricks)• Implemented auto integration between Azure Active Directory and Data Lake, streamlining user access and permissions.• Created data and analytical models for all zones within the data lake, optimizing data utilization and analysis.• Developed a comprehensive package utilizing predefined client inputs, deploying robust batch processing, and streaming pipelines for multiple sources.• Proficiently managed Azure Data Factory (ADF) pipelines and integrated with various Azure services, ensuring seamless data management and delivery.

• Developed and maintained AWS Data Lake, Data warehouses, databases, and reporting environments. Extensively worked with complex data models to build dashboards.• Created Views, Stored Procedures, User defined functions and optimized large queries.• Monitor, schedule, and manage data pipelines in Apache airflow.• Extensive hands-on experience in managing AWS Glue Pipelines and integrating it with other AWS Services.• Design, build, test, and maintain data pipelines. ETL processes and data management delivery within a s3 global data lake and Redshift environment within AWS Solutions using Glue Jobs, Triggers and Workflows.• Built strong relationships with the data scientist, administration, Stakeholders, and customers to provide weekly presentations, KPI reporting, and dashboards.• Wrote custom scripts to link and consume huge complex raw nested JSON data from the web. Converted the files to parquet format and loaded them into S3 to query Amazon Athena to perform server less operations.• Design and build API integration to support the needs of analysts and reporting systems.• Glue was used to complete ETL procedures. Identify any data biases and provide a report. Identify any data biases and provide a report.

• Designed and constructed a scalable web application that converts natural language text into SQL queries on a procurement database, essentially a Google-like search engine for a self- database.• Pulled, organized, and analyzed relevant data from multiple sources, using tools such as Tableau, VS Code, and programming languages such as SQL and Python• Designed, build, and launch data pipelines to build data environment that generate business insights.• Build dashboards and data infrastructure towards a coherent tech and data ecosystem.• Wrote and deploy SQL to archive and or purge data from databases and to locate, review, explain and document data for retention requirements.• Resolved DE Duplication, Canonicalization, and Record-linkage by performing entity Resolution on a procurement datasets using AWS Glue. Corrected mapping issues and errors throughout the Data flow• Worked with programmers, engineers, and management heads to identify process improvement opportunities, propose system modifications, and devise data governance strategies.• Prepared final analysis reports for the stakeholders to understand the data-analysis steps, enabling them to take important decisions based on various facts and trends.• Worked with programmers, engineers, and management heads to identify process improvement opportunities, propose system modifications, and devise data governance strategies.