In my role as a Data Engineer at Hexaware Technologies, I harnessed Spark SQL in Databricks to extract data from Azure Synapse and ADLS, facilitating the creation of analytical reports. This led to enhanced decision-making processes and improved data-driven insights for end users. • Developed analytical analysis in collaboration with stakeholders.• Utilized Spark SQL in Databricks to extract data from Azure Synapse and ADLS.

• Seamlessly executed data ingestion and processing operations using Apache Spark, interfacing with Snowflake and S3, thereby streamlining data workflows and accelerating processing time for extensive datasets.• Automated large data modeling tasks using EMR Cluster with Spark, resulting in a 50% reduction in data processing time, enhancing data quality.• Used AWS Lambda functions for automating data processing, leading to a 34% improvement in efficiency.• Developed and maintained complex ETL workflows in Airflow, scheduling and monitoring data processing tasks to ensure timely and accurate data delivery across various systems.• Achieved almost 99% uptime for the database system by implementing database replication for high availability.• Skilled in developing comprehensive data models to optimize data storage and retrieval processes.• Skilled in using Spark Data Frame persistency and caching mechanisms to reduce data processing overhead and improve query performance.• Skilled in data profiling and management using SQL queries, with a focus on creating, testing, and refining queries to support data profiling and optimizing data quality and integrity

• Created integration objects to ingest data from external storage sources like S3 for efficient loading into Snowflake. Performed necessary transformations using dbt, and integrated with GitHub for version control.• Automated data ingestion processes between S3 and Snowflake using Event triggers and Snowpipe, improving data availability and freshness for critical business dashboards and decision-making.• Proficient in utilizing AWS Glue Catalog for effective metadata management and data cataloging, and performed quick data analytics using Amazon Athena, enabling fast and efficient querying of large datasets stored in S3.• Implemented and optimized RESTful API to seamlessly integrate diverse data sources, enhancing efficiency and reliability.• Managed and analyzed large volumes of log data daily using Splunk, resulting in proactive issue resolution and a notable improvement in incident response times.• Experienced in utilizing a data warehouse to store and retrieve structured and unstructured data for analysis and reporting.• Reduced manual workflows by 20% by converting business requirements into technical specifications, demonstrating strong communication skills and project management skills in a cross-functional team.• Utilized SQL for thorough data analysis, enabling business intelligence insights and informed decision-making processes.• Applied advanced data modeling techniques using SQL to design comprehensive data models, optimizing data storage and retrieval processes.

• Completed extensive training in Python, SQL, and Data Structures, alongside in-depth immersion in Big Data technologies, focusing on Apache Spark, Hadoop, and cloud-based data platforms• Participating in data engineering projects such as building data warehouses or designing databases.• Contributing to implementing data transformations to support business objectives and analytical needs.

• Conducted data collection and preprocessing for network performance metrics within BSNL infrastructure.• Collaborated closely with cross-functional teams to design and deploy innovative network monitoring tools, leveraging advanced analytics to enhance reliability and performance. • Assisted in the documentation of procedures and protocols for network management, ensuring streamlined operations and effective communication across departments.