• Around 7+ years of extensive experience as a Machine Learning / Data Engineer and Big data Developer specialized in Big Data Ecosystem-Data Ingestion, Modeling, Analysis, Integration, and Data Processing. • Extensive experience in providing solutions for Big Data using Hadoop, Spark, HDFS, Map Reduce, YARN, Hive, Sqoop, HBase, Oozie.• Strong experience working with Amazon cloud services like EMR, Redshift, DynamoDB, Lambda, Athena, Glue, S3, API Gateway, RDS, CloudWatch for efficient processing of Big Data.• Experience working with varied forms of data infrastructure inclusive of relational databases such as SQL, Hadoop, Spark, and column-oriented databases such as MySQL. • Experience in developing Spark applications using Spark - SQL in Databricks for data extraction, transformation, and aggregation from multiple file formats for analyzing & transforming the data to uncover insights into the customer usage patterns.• Experience working with Snowflake Multi cluster and virtual warehouses in Snowflake.• Expertise in creating Spark Applications using Python (PySpark) and Scala.• Proficiency in data warehousing inclusive of dimensional modeling concepts and in scripting languages like Python, Scala, and JavaScript. • Hands on experience building PySpark applications for batch and stream processing involving Transformations, Actions, Spark SQL queries on RDD’s, Data frames and Datasets.• Strong experience writing, troubleshooting, and optimizing Spark scripts using Python.• Strong knowledge on performance tuning of Hive queries and troubleshooting various issues related to Joins, memory exceptions in Hive.• Exceptionally good understanding of partitioning, bucketing concepts in Hive and designed both Managed and External tables in Hive.• Experience in importing and exporting data between HDFS and Relational Databases using Sqoop. • Migrated some projects from Azure Databricks to Synapse Analytics.• Experienced in building highly databricksble Big-data solutions using NoSQL column-oriented databases like Cassandra, MongoDB and HBase by integrating them with Hadoop Cluster.• Extensive work on ETL processes consisting of data transformation, data sourcing, mapping, conversion and loading data from heterogeneous systems like flat files, Excel, Oracle, MSSQL Server.