• Over all 12+ Yrs experience in software engineering. 6-year experience as Hadoop/ETL Developer and AWS engineer.• 3+ years of Experience in PySpark with expertise in Spark SQL queries• Experience in NoSQL(mongodb), DataLake • Experience working with huge volumes of data, analyzing record sets for Data Quality, Data Validation, and Metrics using SQL/ NoSQL, Unix, Data stageDOP, dropping/rebuilding the indexes, and gathering stats on the tables/partitions.• Extensively worked on Python Programming for Automation and connecting different ecosystems• Robust experience in Niche technologies like Apache Spark for Fast Data Processing• Best working experience with Big Data Technologies like Hadoop, Hive, YARN, Spark, Datasets, Streaming• Strong Experience in Data Science programming and ecosystems• Implemented Pipelines to add and get data from Amazon S3. Created Power BI Reports using the Tabular SSAS models as source data in Power BI on-prem and cloud and published reports to service. Implemented multiple bash scripts using Amazon CLI• Implemented Kafka to handle real-time data. Creating the topics, Connect with Consumer to pull the data from sources and load the AWS real-time data in S3• Designed and implemented CI/CD pipelines and Confluence using AWS Code Pipeline, Code Build, and Code Deploy, reducing deployment• In Depth involvement in Data Movement, Data Transformation and Data Engineering from Start to End• Designed and implemented AWS-based data solutions using services like Amazon EMR, Amazon S3, and Amazon Redshift. Loaded the data into Simple Storage Service (S3) in the AWS Cloud.• Impressive expertise in writing Hadoop Jobs for analyzing data using Hive Query Language• Solid experience with various Hive performance tuning techniques like Partitioning and Bucketing.• Experience in importing and exporting data between HDFS and RDBMS using SQOOP• Possess High Enthusiasm for penetrating into the uncharted world of Artificial Intelligence like Deep Learning• Exceptional skills in Automating repetitive tasks using Python, UNIX Bash Shell Scripting, AutoSys Job Scheduler• Highly proficient in writing Database scripts like SQL Scripts which include Stored Procedures, Functions, Triggers, Packages, Indexes• Solid Experience in Investment Banking, Insurance, Telecom, and Government Sector Domains