● Around 10+ years of professional IT experience in BIG DATA using HADOOP framework and Analysis, Design, Development, Documentation, Deployment and Integration using SQL and Big Data technologies as well as with cloud technologies AWS, AZURE and GCP. ● Strong experience developing Spark applications using Spark SQL, Pyspark, and Delta Lake in Databricks for data extraction, transformation, and loading from multiple file formats for visualization and analysis.● Competency in designing pipelines to extract data from a variety of sources, transforming data according to analytics requirements using Data Flow, and loading refined data to desired destinations in Azure Data Factory (ADF). Expert in performing incremental loading in Data Factory using control table and watermarks.● Extensive use of cloud computing infrastructure such as Amazon web services (AWS), Azure and GCP.● Expertise in writing scripts in Python which will create airflow jobs for automating the data pipelines● Familiar with dimensional modeling and strong hands-on experience with Star Schema and Snow-Flake Schema for the fact, and dimension tables in Data Warehouse.● Experience in using messaging queues like Apache Kafka.● Vast expertise in setting up, maintaining, and administering Amazon Web Services (AWS) features like EC2, S3, Redshift, EMR, Glue, Lambda and Athena● Created tables and views in Snowflake to perform data validations●Skilled in T-SQL query performance optimization under SQL Server Management Studio using Tuning Advisor, Execution Plan, Trace Flags, and Extended Events.● Proficient use of Python, PySpark to create Spark applications for interactive analysis, batch processing, and stream processing, understanding of using MapReduce applications and Avro tools to process Avro data files.● Knowledge of (PL/SQL), database design, data analysis, data modelling, data migration, data refresh, and performance tuning.● Has used Kafka and Kafka brokers to start spark context and streaming.● Proficient use of several ETL technologies, such as Informatica Power Centre, for data migration, data profiling, ingestion, data cleaning, transformation, import, and export.● An in-depth familiarity with NoSQL including MongoDB, PostgreSQL, HBase, and Cassandra.● Extensive knowledge and hands on experience with Python, PySpark, Scala, SQL, PL/SQL, and Restful web services.● Expertise in creating unique UDFs for Pig and Hive that incorporate Python/Java functionality into Pig Latin and HQL (HiveQL), as well as experience using UDFs from the Piggybank UDF Repository.