Responsible for designing and implementing effective ETL solutions for structured and unstructured data using Data-bricks ( PySpark / Scala), Data Building Tool on AWS/Azure Infrastructure.Building a data catalog to store metadata, delta tables and enable data governance for Delta Lake.Utilized data analytics to identify trends and patterns in data and provide insights to stakeholders.Creation of data models and documented data flows to improve system architecture and data quality.Creation of Data-bricks Asset Bundles to deploy code to different instances and GitHub integration for version control.Optimizing data warehouse performance through indexing and query optimization

Design Develop and maintain scalable and sustainable data products for custom ETL processes and automation in Azure Data Factory, Databricks and PySpark.Responsible for Creation, Maintenance and Debugging of ETL Pipelines using Azure Data Factory to ingest and transform data from different types of sources.Experience of working with SQL/Azure SQL, Azure Synapse DB and Cosmos DBDeveloping PySpark notebooks in data bricks for data extraction, transformation and aggregation for multiple file formats to uncover insights into customer usage.Good understanding of Spark architecture including spark core, Spark SQL, Data frame, Spark streaming, driver-worker node, data warehousing, data modeling and data integration concepts.Maintain Documentation of Data Pipelines and Workflows for knowledge sharing.Interact and collaborate with stakeholders to understand functional requirements and translate them into technical solutions. Provide technical guidance and mentorship to junior developers to build fault tolerant applications and fostering professional growth.

Understanding Client Requirements and translate them into Technical/Functional requirements for Data Modelling team.Perform data-oriented implementation tasks to ingest data, provisioning data storage, Implement security requirements.Perform Data Quality Validation, Transformation the data using Databricks notebooks.Create ER diagrams for Fact and Dimension Tables and write relational database queries.Configure Data Lake Storage for maintaining Raw, Curated and Processed Data, creation of JSON configuration files for processing data into medallion layers of storage. Worked on creating stored procedures, functions, triggers, indexes.Extensive Domain knowledge of Insurance sector.

Created and maintained project schedules that identified tasks, dependencies, deliverables and timelines.Gathering Client Requirements and Translate them into Technical/Functional Requirements.Defining Project Scope, Project Plan, Risk Management, creation of weekly and monthly status report and present to Product Owner.Backlog Creation, User Story creation and Team Effort Distribution on Rally and Jira.Holding Scrum Ceremonies, facilitating meetings, daily stand-ups, sprint planning, and retrospectives.Creation and enhancement of Data-bricks Notebooks to ingest data from Salesforce to Data Lake.Monitoring Azure ETL Pipelines and debugging failures.

Single point of contact for major Service Now Incidents & Requests. RCA and Debugging issues in existing applications and development of new Applications. Developing new PL/SQL packages, Interfaces, Conversions and Reports (RDF, BI Publisher) Performing Unit, Regression, System, Interface and User Acceptance Testing. Creating Technical Design Documents, Unit Test Plan and Co-ordinate release management for Monthly and Weekly releases.Migration of bug fixes & newly developed components to various instances and Post Deployment Support.Developing Concurrent programs, ESS Jobs Setup, Complex PL/SQL Packages.Maintaining joint contact guide, shift schedules, training new team members and assisting junior developers.

Requirement Gathering & Analysis, Creation of Technical/Functional Design documents (MD070 /MD50), Unit Test Plan. Development and Enhancement of components (Coding, Unit Testing, Integration, Release, Post Deployment Support).Writing Pseudo Code, PLSQL packages, procedures and functions, Debugging and RCA.Migration of Deliverables between different instances, install file creation.Development and Customization of Oracle Forms & Reports.Developing Concurrent programs, ESS Jobs Setup, Complex PL/SQL Packages.Developing Inbound and Outbound interfaces and Conversions.