• Hands on experience in Python PySpark programming on Cloud era. Harton Works and MapR Hadoop Clusters. Aws EMR clusters, AWS Lambda functions and CFTS• Optimize Pyspark scripts to run on Palantir DEEP Env for faster data processing• Work closely with project Business Analyst. Data Modeler and BI Lead to ensure that the end to end designs meet the business and data requirements• Involved in story-driven Agile development methodology and actively participated in daily Scrum meetings• Conducted Data blending, Data preparation using Alteryx and SQL for Tableau consumption and publishing data sources to Tableau server.• Handle importing of data from various data sources; perform transformations using Hive, impala, Map Reduce, load data into HDFS and extract the data from Oracle into HDFS using Sqoop. • Experience in moving data between GCP and Azure using Azure data factory.• Design and Development of various data flows from different source systems to various cloud service providers using Python, Airflow and Apache Nifi• Launched multi-node kubernetes cluster in Google Kubernetes Engine (GKE) and migrated the dockerized application from AWS to GCP• Managed large datasets using Pandas and da - gcp package and MySQL • Worked with Hadoop, SSIS, SSAS OLAP cubes, Pentaho. Hadoop, PIG, Hive, Spark, Oracle, and MS SQL Server• Hands-on experience in using Google Stack driver for monitoring the logs of both GKE and GCP instances and configured alerts from Stack driver.• Develop the Spark Sql logics which mimic the Teradata ETL logics and point the output Delta back to Newly Created Hive Tables and as well the existing TERADATA Dimensions, Facts, and Aggregated Tables.• Setting Up AWS and Microsoft Azure with Databricks, Databricks Workspace for Business Analytics, Manage Clusters in Databricks

• Experience in MVC and Micro Services Architecture with Spring Boot and Docker, Swamp.• Worked on optimization of Hive queries using best practices and right parameters and using Hadoop, YARN, Python, and PySpark.• Was involved in setting up of apache airflow service in GCP. • Working exposure to CA TDM utilities Javelin, CA TDM Portal, Agile Requirement Designer (ARD).• Created jobs in continuous integrated build and testing and deployment using Jenkins Maven• Good experience in methodologies like Waterfall. Agile and Scrum processes.• Developed PySpark Data Ingestion framework to ingest source claims data into HIVE tables by performing Data cleansing Aggregations and applying De-dup logic to identify updated and latest records• Good knowledge in Graph databases Janus graph and Neo4j• Experienced in Ab Initio (GDE 3.3.2) and UNIX Programming• Connecting Apache DD to Kafka for performing analytics.• Worked on python PySpark programming• Performed custom data conversions and developed custom software for risk management applications including . Commercial insurance, life insurance, etc. tracking. Perform all data field mappings.• Created UNIX scripts to execute Oracle Packages thru Control-M jobs.• Evaluate Snowflake Design considerations for any change in the application.

• Loading data from different source(database & files) into Hive using Talend tool.• Involved in collecting, aggregating and moving data from servers to HDFS using Apache Flume.• Objective of this project is to build a data lake as a cloud based solution in AWS using Apache Spark.• Developed Pig scripts to parse the raw data, populate staging tables and store the refined data in partitioned DB2 tables for Business analysis.• Worked on managing and reviewing Hadoop log files. Tested and reported defects in an Agile Methodology perspective.• Conduct/Participate in project team meetings to gather status, discuss issues & action items• Provide support for research and resolution of testing issues.• Implemented Installation and configuration of multi-node cluster on Cloud using Amazon Web Services (AWS) on EC2.• Extensive experience in writing UNIX shell scripts and automation of the ETL processes using UNIX shell scripting.