Data Engineer 2
Current- Working on DIME(Data Insights Automation Engine) which is a comprehensive toolkit designed to make the entire lifecycle of data insight creation and management more scalable and efficient.- E2E Design (HLD+LLD) for Alerting & Telemetry Module - Leveraging this module data publishers visibility into the overall performance of data pipelines and get notified of issues before they affect downstream processes.- Worked on Query Templatization Service to port service to spark engine, since… Show more - Working on DIME(Data Insights Automation Engine) which is a comprehensive toolkit designed to make the entire lifecycle of data insight creation and management more scalable and efficient.- E2E Design (HLD+LLD) for Alerting & Telemetry Module - Leveraging this module data publishers visibility into the overall performance of data pipelines and get notified of issues before they affect downstream processes.- Worked on Query Templatization Service to port service to spark engine, since Data Warehouse was not able to scale causing slow execution , high cost and below par reliability. Show less