" Implemented Data Lake and ETL Pipelines for leading Aircraft Manufacturer " where I have: ● Developed ETL pipelines to transfer data from 3rd party sources like ALB and FCO to the curated layer of Skywise tools, leveraging Python, PySpark, and SQL.● Created and optimized data ingestion workflows to inject data from external applications into the Foundry/Skywise tool.● Assisted in developing, designing, and implementing data products and datasets leveraging PySpark… Show more " Implemented Data Lake and ETL Pipelines for leading Aircraft Manufacturer " where I have: ● Developed ETL pipelines to transfer data from 3rd party sources like ALB and FCO to the curated layer of Skywise tools, leveraging Python, PySpark, and SQL.● Created and optimized data ingestion workflows to inject data from external applications into the Foundry/Skywise tool.● Assisted in developing, designing, and implementing data products and datasets leveraging PySpark, SQL, Linux, and Python to fulfill client needs.● Designed an ontology object on the dataset to build and sustain a detailed data model, supporting improved business processes and analytics.● Refactored Java code into Python to enhance AWS resource efficiency and increase code readability. Show less

Accomplished "Data Migration Solutions for leading Telecom Company" where I:● Developed an ETL data pipeline from Oracle and Hive (on-premise) to Snowflake, using AWS Glue, Athena, RDS, Airflow, and PySpark.● Worked on the creation and ingestion of tables via Snowflake stored procedures and functions, aimed at facilitating data visualization.● Migrated approx. 600 tables from Exadata to Snowflake using AWS Glue, and managed scheduling and orchestration with… Show more Accomplished "Data Migration Solutions for leading Telecom Company" where I:● Developed an ETL data pipeline from Oracle and Hive (on-premise) to Snowflake, using AWS Glue, Athena, RDS, Airflow, and PySpark.● Worked on the creation and ingestion of tables via Snowflake stored procedures and functions, aimed at facilitating data visualization.● Migrated approx. 600 tables from Exadata to Snowflake using AWS Glue, and managed scheduling and orchestration with Airflow.● Leveraged AWS Lake Formation to manage PII column access in use case tables, ensuring data is masked from unauthorized users. ● Extracted the DDLs of around 450 tables from on premise system and created the same in snowflake.● Performed hashing on AWS and Snowflake tables to anonymize PII attributes for join operations, allowing users to view only dummy attributes.● Managed job creation and daily monitoring of scheduled Apache Airflow jobs, taking appropriate recovery actions in case of failures. ● Established data ingestion to S3 buckets and Snowflake tables with AWS Glue and RDS.● Enhanced data processing efficiency by 30% through ETL pipeline optimization upon project completion. Show less

" Implemented ETL Data Solutions for Premier Indian Bank " where I have : ● Designed a data pipelines for ingestion, transformation, and loading to the standard layer; streamlined processes reduced manual intervention by 40 hours per month, allowing data scientists to focus on analysis. ● Handled data ingestion from Oracle and SQL Server databases on a daily basis, focusing on data cleaning, joining, aggregations, and migration to the standard layer.● Played a… Show more " Implemented ETL Data Solutions for Premier Indian Bank " where I have : ● Designed a data pipelines for ingestion, transformation, and loading to the standard layer; streamlined processes reduced manual intervention by 40 hours per month, allowing data scientists to focus on analysis. ● Handled data ingestion from Oracle and SQL Server databases on a daily basis, focusing on data cleaning, joining, aggregations, and migration to the standard layer.● Played a key role in automating approximately 350 tables from source to standard data layer using PySpark and Autosys tools.● Built a robust, automated data pipeline in 15 months, ensuring it was free of bugs. Configured email alerts for pipeline failures, and conducted log reviews and necessary recovery actions. Show less