• Responsible for moving data between different AWS compute and storage services by using AWS Data Pipeline.• Used data pipelines for different events to load the data from MySQL and Oracle source to AWS S3 bucket and then into AWS Redshift and delivered high success metrics.• Responsible for using Spark, AWS Athena, Python, and Airflow for data warehouse applications to maintain large datasets in AWS S3• Participate in requirement gathering and analysis phase of the project in documenting the business requirements by conducting workshops/meetings with various business users.• Handled Business logics by backend Python programming to achieve optimal results.• Created Database tables, functions, stored procedures and wrote prepared statements using PL/SQL.• Used AWS EMR for transforming and moving huge data of AWS data stores and data bases like AWS S3, Dynamodb.• Implemented code using Python and SQL to transform and send data to DataLake.• Responsible for loading Data pipelines from web servers using Kafka and Spark Streaming API.• Responsible for using Kafka for real time data ingestion• Performed troubleshooting, fixed, and deployed many Python bug fixes of the two main applications that were a main source of data for both customers and internal customer service team.• Analyzing data in data lake to transform and analyze data on AWS QuickSight using filters, parameters, and calculated fields.

• Worked closely with clients to gather requirement, perform analysis and design solution from a Warehouse standpoint.• Used ER Studio for creating ER diagrams and logical model structures for Oracle and Teradata as per business requirements.• Worked on Sqoop for importing Data from various database systems such as MySQL, DB2 etc., to Data Lake Raw Zone.• Designed and developed solutions to process the Data-at-Rest utilizing Big Data through Hadoop, Map Reduce Frameworks, HBase, and Hive.• Implemented AWS Lambda to perform data validation, filtering, sorting, and other transformations for every data change in a DynamoDB table and load the transformed data to another data storage with heavy user experience.• Worked on predictive and what-if analysis using Python from HDFS and successfully loaded files to HDFS from Teradata and loaded from HDFS to HIVE. • Worked on AWS kinesis data and Amazon Redshift, created data models and extracted Metadata from Amazon Redshift, AWS, and Elastic Search engine using SQL queries to create reports.• Have hands on experience in fact dimensional modeling (Star schema, Snowflake schema), transactional modeling and SCD (Slowly changing dimension).• Used Spark/PySpark Data frames, Spark-SQL, Spark MLLib extensively and developing and designing POC's using Scala, Spark SQL and MLlib libraries, E-commerce.

• Experience in Developing Spark applications using Spark/PySpark - SQL in Databricks for data extraction, transformation, and aggregation from multiple file formats for analyzing & transforming the data to uncover insights into the customer usage, consumption patterns, and behavior.• Skilled dimensional modeling, forecasting using large-scale datasets (Star schema, Snowflake schema), transactional modeling, and SCD (Slowly changing dimension).• Developed scripts to transfer data from FTP server to the ingestion layer using Azure CLI commands.• Created Azure HD Insights cluster using PowerShell scripts to automate the process.

• Involved e2e phases of data mining; data collection, data cleaning, developing models, validation, visualization, performed Gap analysis and provide feedback to business team on areas to improve the product delivery.• Performed Data Mining on bulk datasets of Structured and Unstructured data, Data Acquisition, Data Validation, Predictive modeling, Data Visualization on provider, member, claims, and service fund data.• Developed RESTful services using Python Flask framework, package it in docker and deploy thru Jenkins to Kubernetes.• Experience in building and architecting multiple Data pipelines, end to end ETL and ELT process for Data ingestion and transformation in GCP, AWS and Spark.• Leveraged cloud and GPU computing technologies for automated machine learning such as AWS, GCP.• Created reusable Rest API's to share data from variety of data sources to serve and resolve some of the impeding client needs. • Developed and designed system to collect data from multiple platforms using Kafka and then process it using spark.• Created modules for spark streaming in data into Data Lake using Spark and Worked with different feeds data like JSON, CSV, XML and implemented Data Lake concept.• Worked on the development of Data Warehouse, Business Intelligence architecture that involves data integration and the conversion of data from multiple sources and platform.

• Worked with the project that follow SDLC-agile model.• Worked very close with Data Architects and DBA team to implement data model changes in database in all environments.• Handled day to day production issues effectively by analyzing the root cause of the issue and using effective debugging strategies.• Analyze various data sources such as SQL server, Oracle, Flat files etc. and identify potential source systems for projects.• Developed data models using Erwin tool across the subject areas based on the specifications and established referential integrity of the system.• Reverse Engineering models from Databases, files, External metadata using Erwin.• Maintain and enhance data model with changes and furnish with definitions, notes, reference values and check lists.