• Involved in complete Big Data flow of the application starting from data ingestion upstream to HDFS, processing the data in HDFS and analyzing the data and involved.• Demonstrated expert level technical capabilities in areas of Azure Batch and Interactive solutions, Azure Machine learning solutions and operationalizing end to end Azure Cloud Analytics solutions. • Day to-day responsibility includes developing ETL Pipelines in and out of data warehouse, develop major regulatory and financial reports using advanced SQL queries in snowflake. • Developing Json Scripts for deploying the Pipeline in Azure Data Factory (ADF) that process the data using the Cosmos Activity. • Designed, developed data integration programs in a Hadoop environment with NoSQL data store Cassandra for data access and analysis. • Implement One Time Data Migration of Multistate level data from SQL server to Snowflake by using Python and Snow SQL.

• Designed and Developed Real Time Stream Processing Application using Spark, Kafka, Scala and Hive to perform Streaming ETL and apply Machine Learning. •Led cross-functional Agile teams in the successful delivery of multiple software projects, including web applications and mobile apps, using Scrum methodology • Used Airflow for scheduling the Hive, Spark and MapReduce jobs. • Developed reusable objects like PL/SQL program units and libraries, database procedures and functions, database triggers to be used by the team and satisfying the business rules• Developed Spark code using Scala and Spark-SQL/Streaming for faster processing of data. • Developing Spark programs with Python and applied principles of functional programming to process the complex structured data sets. • • Data sources are extracted, transformed and loaded to generate CSV data files with Python programming and SQL queries. • Worked with Hadoop ecosystem and Implemented Spark using Scala and utilized Data frames and Spark SQL API for faster processing of data.

• Involved in using Terraform migrate legacy and monolithic systems to Amazon Web Services and developed environments of different applications on AWS by provisioning on EC2 instances using Docker, Bash and Terraform.• Managed AWS infrastructure as code using Terraform and used scripts to automate instances. • Used Hive SQL, Presto SQL, and Spark SQL for ETL jobs and using the right technology for the job to get done. • Defined facts, dimensions and designed the data marts using the Ralph Kimball's Dimensional Data Mart modeling methodology using Erwin. • Defined and deployed monitoring, metrics, and logging systems on AWS. • Worked publishing interactive data visualizations dashboards, reports /workbooks on Tableau and SAS Visual Analytics. • Created Data Quality Scripts using SQL and Hive to validate successful das ta load and quality of the data. Created various types of data visualizations using Python and Tableau. • Implemented data streaming capability using Kafka and Talend for multiple data sources.

• Involved in complete Software Development Lifecycle (SDLC). • Worked on different dataflow and control flow task, for loop container, sequence container, script task, executes SQL task and Package configuration. • Extensive use of Expressions, Variables, Row Count in SSIS packages • Created SSIS packages to pull data from SQL Server and exported to Excel Spreadsheets and vice versa. • Created batch jobs and configuration files to create automated process using SSIS. • Data validation and cleansing of staged input records was performed before loading into Data Warehouse • Automated the process of extracting the various files like flat/excel files from various sources like FTP and SFTP (Secure FTP) • Interacted with Business analysts and stake holders to gather business requirements