Developed and deployed data ingestion and storage solutions utilizing AWS S3, Redshift, and Glue, enhancing data management and storage capabilities.Integrated AWS services, including Aurora and DynamoDB, with GCP for seamless data transfer and interoperability between cloud platforms, ensuring data consistency and accessibility.Led implementation of Amazon Redshift for advanced data warehousing solutions, integrating it with Aurora and DynamoDB to consolidate and analyze data from diverse sources.Developed and enhanced ETL workflows using AWS Glue to extract, transform, and load data from AWS S3, Aurora, and DynamoDB into Redshift, optimizing data processing efficiency and ensuring seamless integration.Developed and enhanced ETL workflows using AWS Glue to extract, transform, and load data from diverse sources into Redshift, optimizing data processing efficiency and ensuring seamless integration.Designed and implemented ETL workflows with AWS Glue to efficiently extract, transform, and load data from diverse sources into Redshift, optimizing data integration processes.Utilized AWS EMR (Elastic MapReduce) to leverage the power of Hadoop and SPARK for efficient big data processing and analytics.Utilized AWS SNS and SQS to integrate real-time event processing and messaging capabilities.Utilized AWS Athena to enable ad-hoc data analysis and querying of data stored in Amazon S3, providing on-demand data exploration capabilities.Developed and implemented real-time data processing solutions using AWS Kinesis for seamless data streaming and analysis.Deployed strong IAM policies and roles to guarantee secure user access and granular permissions for AWS resources.Experience in designing and implementing Iceberg tables for storing large-scale structured data in cloud object stores or Hadoop Distributed File System (HDFS).Utilized Typescript with AngularJS to build responsive and feature-rich front-end components for web applications.

Designed and implemented data ingestion and storage solutions using AWS S3Implemented configuration and optimization techniques for Redshift clusters to maximize data processing performance and streamline query execution, resulting in high-performance data analytics capabilities.Incorporated AWS SNS and SQS to facilitate real-time event processing and effective messaging, enabling seamless communication and efficient handling of events.Deployed AWS Athena for on-demand data analysis and querying of data stored in AWS S3, enabling ad-hoc exploration and efficient data retrieval.Created and executed data streaming solutions leveraging AWS Kinesis, facilitating real-time data processing and analysis capabilities.Implemented SPARK using Scala and SPARK SQL for faster testing and processing of data.Utilized AWS CloudWatch to monitor and handle resources, configure alarms, and gather metrics.Leveraged Amazon EMR to efficiently process and analyse large-scale data sets, utilizing its scalable and distributed processing capabilities for enhanced data processing and analytics.Deployed robust IAM policies and roles to establish secure user access and permissions for AWS resources, guaranteeing comprehensive security measures and controlled resource management.Worked with big data technologies including HDFS, Impala, Spark, and Spark SQL to process and analyze large volumes of data. Conducted performance tuning and optimization of big data applications, improving query performance and reducing processing time.Designed, build and managed ELT data pipeline, leveraging AIRFLOW, python, DBT, Stitch Data and GCP solutions.Established CI/CD pipelines using Bamboo, Bitbucket, and GitHub Actions for automated deployments.Designed and fine-tuned data processing pipelines utilizing Hadoop ecosystem technologies including HDFS, Sqoop, Hive, MapReduce, and SPARK, ensuring optimal performance and efficiency in data processing tasks.

Responsible for batch processing of data sources using Apache SPARK and Pig.Work on generating various dashboards in Tableau server using various data source.Analyze, Design, and develop Data discovery dashboards in Tableau to present the data story to the business stakeholders.Developed Tableau data visualization using Cross tabs, Heat maps, Box and Whisker charts, Scatter Plots, Geographic Map, Pie Charts and Bar Charts and Density Chart.Developed SPARK Programs using Scala and Java API's and performed transformations and actions on RDD's.Strong understanding of AWS components such as EC2 and S3.Created external tables with partitions using AWS Athena and Redshift.Implemented AWS solutions using EC2, S3, RDS, EBS, Auto scaling groups.Work with other team members in establishing and sharing design and development techniques of the ETL process.Develop scripts to automate the execution of SPARK jobs of Scala using shell scripts under Unix environment.Handled large datasets using Partitions, SPARK in Memory capabilities, Broadcasts in SPARK, Effective & efficient Joins, Transformations.Experience working with different file formats like Apache Avro, Sequence file, JSON, XML and Flat text file.Documented ETL test plans, test cases, test scripts, test procedures, assumptions, and validations based on design specifications for unit testing.Preparing test data, and loading for testing, error handling and analyzing Data in production, finding root causes for errors in data.Worked on a product team using Agile Scrum methodology to Design, Develop, Deploy, and support solutions that leverage the Cloud data platforms.

Crafted, managed, and provided support for SQL Server databases, including proficient data manipulation and upkeep.Contributed to data modeling, both in terms of logical and physical database design.Facilitated seamless integration between front-end systems and the SQL Server back-end.Formulated Stored Procedures, Triggers, Indexes, User-Defined Functions, and Constraints across diverse database entities to achieve precise outcomes.Executed data imports and exports across servers utilizing tools like Data Transformation Services (DTS).Proficiently composed T-SQL statements for data retrieval while actively participating in the performance optimization of T-SQL queries.Orchestrated the seamless transfer of data from various sources (e.g., MS Excel, MS Access, Flat Files) to SQL Server using SSIS/DTS.Provided instrumental support in resolving SQL Reporting Services and T-SQL-related issues, while demonstrating prowess in designing and formatting a variety of report types.Delivered application support via telephone and developed tested Windows command files and SQL Server queries to ensure 24/7 monitoring of production databases.Developed, monitored, and deployed SSIS packages, ensuring effective data flow and integration. Generated comprehensive enterprise reports (SSRS/Crystal/Impromptu) from both SQL Server database (OLTP) and SQL Server Analysis Services Database (OLAP). These reports encompassed diverse features, including grouping, drilldowns, sub-reports, navigation reports (Hyperlink), and more.Demonstrated expertise in working with varied report types such as tables, matrices, charts, and sub-reports.Skillfully crafted Linked reports, Ad-hoc reports, and other report variations to cater to specific requirements, thereby minimizing report repetition.