Near real-time event driven platform using Storm, Kafka, Cassandra and Elasticsearch which lookup and stitch data from other datasets on real-time and load the stitched data into elastic search and BigQuery to solve real time DC operation and reporting need in less than 5 minutes.Roles & Responsibilities:• Designed and developed a generic platform to extract the data in near real-time from multiple source system to stitch the data and load into Elasticsearch.• Processed >4TB of data daily on near real time and load data in target less than• Event failure is reprocessed to achieve 100% data completeness.• Event failure is reprocessed to achieve 100% data completeness.• Mentored junior engineers to strengthen theirtechnical skills and improve overall teamefficiency.• Exhibited strong technical aptitude andapplication expertise resulting in optimized

Near Realtime platform using Kafka connect to support multiple source system and target system to achieve the load time less than 1 minutes between source to target with 100% data completeness.Roles & Responsibilities:• Designed and developed a generic platform to ingest the data in near Realtime with multiple source system and target system.• Data Engineer developer effort reduced by 95% through automation.• Onboarded 250K source tables for data ingestion in near real time with zero development effort.• ETL failure is automated through platform with self-healing features on connector failures.2013-06 - 2018-08• Features like table creation and modification on target is automated on platform reducing developer time by 100% on table/schema maintenance.• Deployment is automated as part of platform, reducing developer time by 100% on new deployment.• 100% data accuracy achieved through platform by handling late arrival data and source deletion through enhanced framework.

Automatic Inbound Scheduling (AIS) is a project that automates the scheduling process of a Purchase Order and enables business to make more accurate decisions in managing inbound flow and will provide one stop solution for all the networks It provides insight about the compliance of deliveries and cases based on appointment date generated by scheduler system and delivery arrived time.BMS is intended to be the centralized platform to bridge the gap between DC operators and HO teams (DC Ops, Merch Ops and Flow), allow for DCs to raise concerns and get them addressed through a digitized workflow. Daily volume Breach Mitigation System is an interactive web application which allows users to raise a concern, assign to decision makers and address the concern and track savings. Interceptor is a diagnosticRoles & Responsibilities:• Designed and led the development of a scalable big data processing platform, handling over >2 TB of data daily, enhancing data analysis capabilities.• Developed and maintained ETL pipelines for data ingestion, processing, distribution, improving data quality, accessibility, and reporting.• Developed 15+ data models that supported business decision-making and increased revenue by 40%.• Reduced data processing time by 90% by optimizing ETL pipelines and SQL.• Designed and migrated 100+ Alteryx data pipelines, improving data flow efficiency by 99%.• Managed and maintained a database of over 10.5 billion records with 99.9% uptime.• Analyzed and developed various reporting Tableau/Looker Dashboard for business insights with increased query response time by 84% through performance tuning.

Designed the Data lake architecture which includes Data Quality, Metadata management, Governance and tools for business access like Tableau, Thoughtspot etc.Evaluating acceleration layers on big data including GPU based technologies.In addition to evaluating different technologies and tools, as Senior software engineer, I have been working on many cross-functional projects related to Data integration across multiple Walmart propertiesMarketplace PaymentsFresh (Grocery) implementationData services implementation for Seller portal360 product Suite (customer 360, order 360, product 360, offer 360, shipment 360)