Senior Data Engineer
CurrentProject: AllyGPO | Domain: Healthcare• Developed ETL pipelines for the Patient Management System, enhancing data ingestion,validation, and loading while ensuring data compliance• Standardized data ingestion with Azure Functions in Python, reducing vendor onboardingefforts by 50%• Refactored Azure Data Factory pipelines, achieving an $800 cost reduction in Azuresubscriptions through optimization• Improved query execution times by 2x in the Data Warehouse using effective storage,partitioning, and indexing strategies• Led a proof-of-concept for Event-Driven Architecture with Event Hubs and Grids, reducingpipeline runs and resource usage by 40%• Managed Azure DevOps for project management, code repositories, and CI/CD deployments,handling various data types (CSV, JSON, Avro, Parquet)• Automated workflows with Logic Apps, configured Azure RBAC policies, and utilized KQL for loganalyticsProject: Netsmart | Domain: Healthcare• Engineered a Lambda-based processing system with Glue Data Catalog, Redshift, andCloudFormation, enhancing patient readmission processes and improving recovery rates by 20%• Established a robust ETL pipeline with integrated data quality checks, ensuring 100% dataintegrity before loading into the warehouse through unit testing• Developed a Kubernetes cluster for autoscaling nodes, optimizing resource allocation based onreal-time demand• Implemented parallel processing of large datasets using Apache Spark in Databricks, reducingprocessing time by 50%• Collaborated on data flow documentation (HLD, LLD, FSD) and unit tests, ensuring high-qualitydeliverables with low defect rates
