Results-driven Data Engineer with 5+ years of experience in designing and implementing robust data solutions to drive business insights and enhance data-driven decision-making. Experience in Evaluation/design/development/deployment of additional technologies and automation for managed services on S3, Lambda, Athena, EMR, Kinesis, SQS, SNS, CloudWatch, Data Pipeline, Redshift, Dynamo DB, AWS Glue, Aurora DB, RDS, EC2. ETL pipelines in and out of data warehouse using Python with Pandas and Spark. Proficiency in building Data pipelines and Data loading using Azure Data Bricks and Azure Data Warehouse to control the accessibility to the database. Proficient in building CI/CD pipelines in Jenkins using pipeline syntax and groovy libraries. Implemented Big Data solutions using Hadoop technology stack, including PySpark, Hive, Sqoop, Avro and Thrift. Practical experience using KTables, Global KTables, and KStreams in Apache Kafka and Confluent environments to work with Kafka streaming. Experience in Cisco Cloud Center to more securely deploy and manage applications in multiple data center, private cloud, and public cloud environments. Good understanding of Spark Architecture with Databricks, Structured Streaming. Setting Up AWS and Microsoft Azure with Databricks, Databricks Workspace for Business Analytics, Managing Clusters in Databricks, Managing the Machine Learning Lifecycle. Highly skilled in using visualization tools like Tableau, matplotlib, ggplot2 for creating dashboards. Experience in automating day-to-day activities by using Windows PowerShell.