Data Engineer
Current• Collaborated on the development of Medicare claims ingestion workflows under new CMS regulations, leveraging Databricks, and Spark cluster for the ingestion of 1 TB claim data using PySpark. Performed data transformations, converting raw data into Parquet. • Developed Azure Data Factory pipelines to orchestrate the entire data flow, ensuring seamless and compliant processing, storage, and reporting of dental and pharmacy claims. • Developed and optimized SQL Server stored procedures to ingest processed data, supporting business-critical functions.• Designed and implemented a Generative AI model using transformer-based architectures to generate contextual summaries of claims, integrating it with existing workflows to assist the business team in faster and more accurate decision-making.• Analyzed, validated, and processed over 1 million healthcare claims, ensuring adherence to CMS requirements and mitigating financial risks by implementing robust compliance mechanisms.