Senior Data Engineer
Current• Currently working in enterprise mobility team as data engineer. As part of this team, I worked on designing and developing batch and streaming data pipeline for enterprise mobile data. Processed data is being consumed at real time for reporting. Data is consumed from Kafka topic at real time, processed and stored in BigQuery for real time analysis.• Designed and developed batch data pipeline for processing network data, which is consumed for building anomaly detection machine learning… Show more • Currently working in enterprise mobility team as data engineer. As part of this team, I worked on designing and developing batch and streaming data pipeline for enterprise mobile data. Processed data is being consumed at real time for reporting. Data is consumed from Kafka topic at real time, processed and stored in BigQuery for real time analysis.• Designed and developed batch data pipeline for processing network data, which is consumed for building anomaly detection machine learning model.• Designed and developed near real time data pipeline for consuming device data for anomaly prediction. Data is consumed from Kafka topic and used by deep learning model for anomaly prediction.• Designed and developed Locus service for devices enrolment in Walmart network and metric collection service for REST API using Golang, Prometheus and Grafana, which is deployed on WCNP.• Developed anomaly detection model for modelling behaviour of all store devices. This system consists of multiple ML model detection anomalous behaviour at multiple network stages. LSTM deep learning model for anomaly detection. Show less