Parth Patel Email & Phone Number

Roanoke, Texas, United States

In my role as a Senior Data Engineer at Mercedes-Benz Financial Services in Roanoke, Texas, since August 2021, I've excelled in various data engineering tasks. This includes leveraging Spark and Scala for high-speed data processing, implementing Big Data Analytics and Machine Learning applications with Apache Spark and Python, and developing Spark Streaming jobs for Kafka data.I've worked extensively within the Hadoop ecosystem, utilized a range of big data tools, converted Hive/SQL queries into Spark transformations, and led cloud migration automation efforts. My responsibilities have encompassed data analytics, reporting, real-time stream processing, and the evaluation and implementation of various big data technologies. This role has allowed me to showcase my proficiency across a diverse spectrum of data engineering tasks and technologies, contributing significantly to data processing and analytics efforts.

St Louis, Missouri, United States

As a Data Engineer at Centene Corporation in St. Louis, Missouri, from Feb 2019 to Jul 2021, I played a significant role in various data engineering tasks. I employed Agile Scrum methodology, configured Apache Airflow for workflow management, and developed Spark scripts using Python. I handled Hadoop production support, Hive table creation, data loading, and optimization through Hive queries and Pig UDFs.My experience extended to Big Data analysis using various technologies including Scala, Spark, Spark SQL, Hive, and more. I worked with SQL and other database components and wrote MapReduce programs and Hive UDFs in Java. I was proficient in Cloudera Data Platform and automation using PowerShell and JSON templates.ETL tasks were performed using Apache NiFi, and data pipelines were created in Azure Data Factory for cloud based ETL. I utilized Power BI for report creation, and Azure services like Azure Data Lake and Azure Databricks for data processing.Furthermore, I worked on real-time streaming with Kafka and Spark Streaming, implemented PySpark batch jobs, and designed ETL frameworks using Java. My role involved extracting data from various sources for analysis, combining SQL queries and PySpark for insights, making me well-versed in diverse data engineering tasks.

Bangalore, India

As a Spark/Big Data Developer at Quotient in Bangalore, India, from Aug 2017 to Nov 2019, my responsibilities included extensive use of Spark for real-time data streaming and transformation. I successfully worked with various data sources, such as Teradata and Oracle, loading data into HDFS and Hive/Impala. I developed Python-based Spark applications for loading numerous CSV files into Hive ORC tables in a distributed environment. I also configured Apache Hadoop, created YARN application dashboards, and handled data streaming with Kafka.Additionally, I optimized Hive queries, wrote HiveQL scripts, and performed real-time processing using Spark Streaming with Kafka. I worked with various data formats, including JSON, ORC, and Parquet, on HDFS using PySpark. I also had experience with custom UDF development in Python, data import/export using Sqoop, and migration to cloud databases like Azure Synapse Analytics and Azure SQL DB. In the cloud, I used Azure Data Factory, T-SQL, Spark SQL, and Azure Data Lake Analytics for ETL processes and implemented Spark using Scala and Python, leveraging DataFrames and Spark SQL for efficient data processing.

New Delhi, India

During my 3-year role as a Big Data Hadoop Developer at Amway Corporation, I managed diverse data sources remotely from New Delhi (2014-2017). I led HDFS maintenance, migrated data from Oracle, Netezza, MySQL to Hadoop, and utilized Java for Map Reduce jobs, ensuring clean data. I analyzed Time Series Data in HBase, performed incremental imports to Hive, and conducted rigorous unit testing.My expertise extended to Microservices, Sqoop, MR2, PIG, Hive for batch jobs, while using Pig Latin for file processing. I optimized Map Reduce algorithms, employed compression techniques, and crafted Java-based UDFs for data insights. I efficiently used Hive for correlating customer browser logs and leveraged Cloudera Manager for workload monitoring.Incorporating Qlik Sense Desktop 2.x, I developed applications, reports, and maintained technical documentation. Throughout, I actively participated in SDLC phases, code reviews, and embraced Agile Methodology in this collaborative and dynamic environment.