Leading a team of 10 members and working with business and development teams to understand the business functionality and (Snowflake/snowpark) technical requirements and POC for the assigned project. Handled and provide views in the preliminary and detail design reviews, Code review.Design the Python program to fetch the data from Data base tables and load the same in Oracle and to validate the target files which are generated from the DB table to load into Snowflake, snowpipe operate the file exceptions by using Python programs.Integrated Teamcenter as a central repository for managing and integrating data from various sources, ensuring consistent and up-to-date information for all stakeholdersEnsuring proper connections between Postgres and AWS, as well as real-time data from AWS to Snowflake. Monitoring transformations and connections between DBT and Snowflake, as well as the Matillion GitHub repository

Involved in implementing and following an Agile development methodology within the cross-functional team.Developed comprehensive test suites using Java, Python and PY Spark to validate data transformations and quality checks at each stage of the data processing with snow pipe.Experience in designing and implementing data lake architectures with Amazon Athena,Conducted database migration to RDS from on-premises databases.Experience in writing and deploying cloud functions on AWS Lambda with Informatica, with DBT.Proficient in designing and implementing complex Spark SQL-based data processing pipelines that involve ETL, and data warehousing using Spark Data Frames and Spark SQL.

Commitment to clear and timely communication with all stakeholders - including Clients, Client Service Managers, Business Analysts, IT Operations, Dev/Engineering team. Conducting Functional, regression and Integration testing to ensure defect free release.

While involving in development activities performing the tasks like code changes asper the standard, code commit, deployment through Jenkins pipeline into non-prod environments, testing the application, updating JIRA story about the status of the task, and creating Pull Request to get it approved and merged to the master branch for production deployment.Processed transactions accurately and efficiently, ensuring that all transactions were completed within the established time frameParticipating in regular production support meetings with development team to understand the issues occurring in the application and the problems they are facing and handlingin meetings of Production turnaround SLAs, from receiving client data to completing processing and making output available in target systems and to target stakeholders.

Developed and maintained ETL workflows using Apache Airflow, orchestrating the data pipeline execution, monitoring, and scheduling tasks with a focus on reliability and scalabilityDesigned and implemented salable data pipelines using Python and PySpark to process and transform large volumes of structured and unstructured data.Developed a data security framework to ensure data privacy and comply with regulatory requirements and onitoring and optimizing the performance of Data Vault structures and ELT processes in Snowflake.