Joshi G Email & Phone Number

• Designed and developed ETL (Extract, Transform, Load) pipelines using AWS services such as Glue, Lambda, and S3 to extract data from various sources, transform it and load it into a data warehouse.
• Build and maintain scalable and highly available data solutions using AWS services such as Redshift, RDS, and DynamoDB.
• Extensively worked with the application data present in AWS S3 storage for performing required transformations.
• Configured and deployed production-ready data… Show more
• Configured and deployed production-ready data pipelines using AWS Glue, S3 and Lambda to maximize data extraction, transformation, and loading.
• Designed and implemented data ingestion pipelines using Apache Airflow to efficiently transfer and process data from various sources.

Worked on Pandas DataFrame operations to clean and preprocess raw data, handling missingvalues and outliers.Developed an ETL process to transform raw data into a structured format using tools such asCloud Dataprep and Cloud Functions.Utilized Apache Beam to construct complex data transformation, ensuring data quality andconsistency. workflowsDeveloped and.

• Migrated the Snowflake database to Windows Azure and updated the Connection Strings based on the requirement.
• Developed Talend MDM jobs to populate the claims data to data warehouse - star schema, snowflake schema, Hybrid Schema.
• Developing Spark applications using Spark-SQL in Databricks for data extraction, transformation, and aggregation from multiple file formats for Analyzing& transforming the data to uncover insights into the customer.
• Creating pipeline… Show more
• Creating pipeline jobs, scheduling triggers, Mapping data flows using Azure Data Factory(V2), and using Key Vaults to store credentials.
• Creating Elastic pool databases and scheduling Elastic jobs for executing TSQL procedures.

• Hands on experience in Spark and Spark Streaming creating RDD & applying operations, transformations, and Actions.
• Used Python with Apache Spark for distributed data processing on Hadoop clusters.
• Used Spark and Spark-SQL to read the parquet data and create the tables in hive using the ScalaAPI.
• Developed Spark code using Scala and Spark-SQL for faster processing and testing.
• Designed and implemented complex data processing pipelines using Scala and Apache… Show more
• Designed and implemented complex data processing pipelines using Scala and Apache Spark.

Master'S Degree, Computer And Information Sciences

Bachelor'S Degree, Electronics And Communications Engineering