• Designed, Developed, and implemented the Azure Data factory framework (V2) with Error logging to populate data in the Azure SQL Data warehouse from Azure Blob storage and Azure data lake store.• Implemented large Lambda architectures using Azure Data platform capabilities like Azure Data Lake, Azure Data Factory, Azure Data Catalog, HDInsight, Azure SQL Server, Azure ML and Power BI. • Experience in Developing Spark applications using Spark - SQL in Databricks for data extraction, transformation, and aggregation from multiple file formats for analysing & transforming the data to uncover insights into the customer usage patterns.• Working experience with data streaming process with Kafka, Apache Spark, Hive. • Developed JSON Scripts for deploying the Pipeline in Azure Data Factory (ADF) that process the data using the SQL Activity.

• Primarily Responsible for converting Manual Report system to fully automated CI/CD Data Pipeline that ingest data from different Marketing platform to AWS S3 data lake.• Experience in developing scalable & secure data pipelines for large datasets.• Migrated and developed the daily running jobs from on-premises to EMR spark, Scala.• Used Data Frame API in Scala for converting the distributed collection of data organized into named columns, developing predictive analytic using Apache Spark Scala APIs• Developed Hive queries to pre-process the data required for running the business process.• Worked on ETL Migration services by developing and deploying AWS Lambda functions for generating a serverless data pipeline which can be written to Glue Catalog and can be queried from Athena.

• Developed a data platform from scratch and took part in requirement gathering and analysis phase of the project in documenting the business requirements.• Worked in designing tables in Hive, MYSQL using SQOOP and processing data like importing and exporting of databases to the HDFS, involved in processing large datasets of different forms including structured, semi-structured and unstructured data.• Developed, built, and deployed applications using Azure stack development kit.• Building and Installing servers through ARM Templates for Azure Portal.• Worked on Ansible, used YAML packages for installing, configuring push to make changes in time• Created the containers using Docker• Built database model, Views and API's using Python for interactive web-based solutions.

• Experience in Big Data Analytics and design in Hadoop ecosystem using MapReduce Programming, Spark, Hive, Pig, Sqoop, HBase, Oozie, Impala, Kafka• Performing hive tuning techniques like partitioning and bucketing and memory optimization.• Worked on different file formats like parquet, orc, json and text files.• Experience in developing scalable & secure data pipelines for large datasets. • Partnered with ETL developers to ensure that data is well cleaned, and the data warehouse is up to date for reporting purpose by Pig. • Supported data quality management by implementing proper data quality checks in data pipelines.• Involved in the data support team as role of bug fixes, schedule change, memory tuning, schema changes loading the historic data.

• Involved in designing physical and logical data model using ERwin Data modeling tool. • Designed the relational data model for operational data store and staging areas, Designed Dimension & Fact tables for data marts. • Extensively used ERwin data modeler to design Logical/Physical Data Models, relational database design. • Created Stored Procedures, Database Triggers, Functions and Packages to manipulate the database and to • apply the business logic according to the user's specifications. • Created Triggers, Views, Synonyms and Roles to maintain integrity plan and database security.