My area of expertise is in the design, development, and management of intricate data structures and pipelines. I have a great deal of experience with data engineering and am adept with technologies like Spark, PySpark, SQL, Python, Scala, Airflow, Databricks, Snowflake, Tableau, Power BI, AWS, and Azure. My specialty is using Spark-SQL and Databricks to execute data extraction, transformation, and aggregation and to extract useful information from a variety of file types. My areas of expertise include automating procedures for smooth data input and movement inside Azure cloud settings, orchestrating complicated operations using Apache Airflow and Oozie, and working with Azure Synapse Analytics, Azure SQL, Azure Data Lake Analytics, Azure Data Factory, and Azure Functions.I have also conducted user training sessions, troubleshooted data visualization difficulties, and optimized Tableau dashboards and reports for optimal performance. I have a strong background in serverless Spark application development using AWS Lambda, managing AWS infrastructure, and putting effective CI/CD pipelines in place. I also know how to create Python-based Map-Reduce flows, optimize Teradata batch processing scripts, and build PySpark-based ETL pipelines. In addition to improving data models for business intelligence tools like Tableau, Power BI, and DBT, I work collaboratively to guarantee efficient project management and productive cross-functional team cooperation. I am also skilled with Jira.