Have worked extensively with AWS services like EC2,S3,VPC,ELB,Auto Scaling Groups, IAM, CloudWatch, CloudFormation, CloudFront, SNS and RDS. Have experience in building and architecting multiple Data pipelines, end to end ETL and ELT process for Data ingestion and transformation. Enhanced and optimized Spark/python jobs to aggregate, group and run data mining tasks using Spark framework.

Was responsible for building scalable distributed data solutions using Hadoop. Involved in Agile Development process like scrum and sprint planning. Also handled hadoop cluster installations in Windows environment. Involved in porting the existing on-premise Hive code migration to GCP BigQuery.

Involved in developing python scripts, informatica and other ETL tools for extraction, transformation, loading of data into Teradata.Worked with a Team of Business Analysts in requirements gathering business analysis and project coordination in creating reports. Performed Unit and Integration Testing of Informatica Sessions, Batches and Target Data.

Creating a web-based application using Python on Django framework for data processing. Implementing the preprocessing procedures along with deployment using the AWS services and creating virtual machines using EC2.Good knowledge in Exploratory data analysis and performed data wrangling and data visualization. Validating the data to check for the proper conversion and identifying and cleaning unwanted data, data profiling for accuracy, completeness, consistency. Preparing standard reports, charts, graphs, and tables from a structured data source by querying data repositories using Python and SQL. Developed and produced a dashboard, key performance indicators and monitor organization performance. Define data needs, evaluate data quality, and extract/transform data for analytic projects and research.