Senior Software Engineer
Current1) Designed and implemented a multi-region-multi-cluster solution for TSDB management utilizing Victoria Metrics, resulting in annual cost savings of ~$20000 by replacing the existing InfluxDB and Prometheus setup.2) Created a distributed logging system with ELFK stack, by consolidating logs from terminals across all DCs and AWS/GCP infrastructure (EC2 + EKS). This facilitated the storage and analysis of approximately 40TB data. Also ensured 0 log loss without adding data replication by… Show more 1) Designed and implemented a multi-region-multi-cluster solution for TSDB management utilizing Victoria Metrics, resulting in annual cost savings of ~$20000 by replacing the existing InfluxDB and Prometheus setup.2) Created a distributed logging system with ELFK stack, by consolidating logs from terminals across all DCs and AWS/GCP infrastructure (EC2 + EKS). This facilitated the storage and analysis of approximately 40TB data. Also ensured 0 log loss without adding data replication by cost optimal engineering practices and processes.3) Created visibility with metrics/alerts/dashboards and distributed the Platform cost b/w all the teams in the org by implementing a Cost-Usage governance framework for all the Data platform components.4) Designed an automated plug-and-play framework for product uptime alerting and reporting on status-page thereby streamlining the uptime reporting process for multiple products in the org.5) Adhering to the principal of least privilege, created an automated multi-level access approval flow, access-grants, revokes for BigQuery access management thereby reducing the TAT from few hours to few minutes.6) Led the project to increase revenue reporting accuracy from ~90% to ~99.9%. Also reduced the revenue report generation runtime from ~12hrs to ~3hrs.7) Developed an Event Delivery Service which consumes over 250M data packets on a daily basis.8) Reduced slot consumption in BigQuery by analysing queries/patterns/usages and coordinating and implementing clustering/partitioning amongst many tables.9) Contributed and maintained an ETL pipeline using Apache Airflow for enabling the syncing of the RDS data from any EC2 machine to BigQuery thereby enabling the analytics on it.10) Created multiple one-stop Looker dashboards for leadership to view all the Engineering metrics, Financial Data reports by creating ETLs from multiple sources and aggregating them on Looker by following best practices. Show less