At Optum Health, my significant role in developing and refining ETL processes has been crucial for the organization's data engineering and machine learning endeavors. My proficiency in Cloud, Python, SQL, and data engineering has been instrumental in advancing data-driven healthcare innovations.A notable accomplishment is the establishment and configuration of an Enterprise Data Lake, which serves as the central repository for the vast amounts of healthcare data. I have engineered intricate ETL pipelines utilizing PySpark and SQL, which ensure the smooth integration and transformation of data from diverse sources. These pipelines are vital for real-time processing and generating actionable insights.My engagement with Cloud services, Reltio MDM has led to the automation and simplification of data workflows, thereby enhancing the efficiency and scalability of large-scale data handling. Moreover, I have instituted security protocols and cost-reduction strategies, thus improving both dependability and cost-effectiveness.In the realm of machine learning, I have managed workflows, streamlining model training and deployment, which results in quicker and more precise forecasts. These models have become integral to Optum Health's bespoke healthcare approach.Furthermore, I have refined the ETL architecture for data migration from OLTP to OLAP systems, guaranteeing that data remains ready for analysis. My efforts with different tools have culminated in swift and dependable reporting mechanisms, providing essential insights to the sales and marketing divisions.Through these initiatives, I have contributed to Optum Health's ability to leverage its data, fostering innovations that improve patient care & operational effectiveness. My enthusiasm for ETL, combined with my expertise in Cloud, Python, & SQL, continues to drive my contributions towards the company's data-centric future.

During my tenure at Albertsons, I played a pivotal role in the development of retail products within an agile team, with a focus on consumer KPIs. I developed a custom, config-driven framework using PySpark, which enhanced reusability and flexibility for various medical products and categories. I also engineered and automated data pipelines to provide centralized KPIs and reports to stakeholders, utilizing tools such as Airflow to automate SPARK, UNIX, and Python scripts.For efficient cloud processing and FHIR loads, I refined scripts and employed Python (with Pandas) and Whistle language to present data in Bigtable, facilitating ad-hoc queries by stakeholders. To ensure data integrity, I established a Data Quality (DQ) framework, and processed and transformed data for analytics within a Big Data ecosystem.Additionally, I developed POCs employing ML models and Cloud ML to analyze table quality in batch processes. My responsibilities expanded to Google Cloud management, including Google Data Catalog, APIs, and BigQuery monitoring. I integrated data from on-premises and cloud sources (MySQL, Cassandra, Azure SQL DB) using Azure Data Factory, executed transformations, and reloaded the data into Azure Synapse.I improved query performance by migrating log storage from Cassandra to Azure SQL Data Warehouse and constructed data ingestion pipelines on Azure HDInsight Spark clusters. Leveraging my Databricks expertise, I facilitated the migration of extensive datasets, crafted notebooks, and performed streaming analytics with Spark Streaming.I automated batch jobs using Azure Logic Apps and extensively engaged with Azure Data Factory for data transformations, Key Vaults, and pipeline migrations. In conclusion, at Albertsons, my passion for data engineering and machine learning has driven impactful solutions that enhance operational efficiency.

• Involved in building the ETL architecture and source to target mapping to load data into Data Warehouse.• Performed Spark Jobs with the Spark core, SparkSQL libraries for processing the data.• Imported data from GCP into Spark Data frame, performed transformations and actions on Data frame.• Experienced in development activities in complete agile model using JIRA and GIT.• Worked with Spark to read the data from Hive and Write it to Bigtable.• Upgrade Map Reduce Programs those are running on the cluster. Involved in loading data from Hadoop file system to Bigtable.• Developed Spark Scripts by using Python, Shell commands as per the requirement.• Experience in Hive Partitioning, Bucketing and Perform joins on Hive tables.• Experience in preparing test data and executing detailed Test plans. Completed required debugging.• Managed end-to-end complex data migration, conversion, and data modeling.• Utilized Kafka to capture and process near Real time Streaming data.• Involved in Writing shell scripts for exporting log files to the Hadoop cluster through the Automated process.• Created monitors, alarms, notifications, and logs for Data using Crontab.• Used Cronjobs End to End data processing pipelines and Scheduling the Workflows.• Managed and implemented Daily continuous code deployment using Jenkins.

• Developed complex transformations in Spark and Hive for calculating sales and returns of the products for forecasting the customer demand. • Sourced Data using spark RDD and Data frame API and optimized the code further to reduce the number of shuffles and improve performance by reducing the latency. • Developed a general-purpose utility in Spark for both In and Out of Data from RDBMS systems to Hadoop HDFS.• Developed the Spark Pipeline to streamline the current processing engine. • Worked with Spark-SQL context to create data frames to filter input data for model execution. • Extensively worked on Hive to analyze the partitioned and bucketed data and compute various metrics for reporting. • Involved in Design and development for building the common architecture for retail data across the Geos. • Designed, and developed spark scripts for parsing the JSON files and storing in Parquet file format in EMR. Big Data analytical solutions using HiveQL and Spark.

• Designed, configured, and deployed Microsoft Azure for a multitude of applications utilizing the Azure stack (Including Compute, Web & Mobile, Blobs, Resource Groups, Azure SQL, Cloud Services, and ARM), focusing on high - availability, fault tolerance, and auto-scaling.