Building SyncLite : Real-Time Data Consolidation Platform, transforming data-intensive application development. (www.synclite.io)SyncLite is an open-source(https://github.com/syncliteio/SyncLite), low-code, comprehensive relational data consolidation platform backed by patented technology. SyncLite excels at performing real-time, transactional data replication and consolidation from a myriad of sources including edge/desktop applications using popular embedded databases (SQLite, DuckDB,… Show more Building SyncLite : Real-Time Data Consolidation Platform, transforming data-intensive application development. (www.synclite.io)SyncLite is an open-source(https://github.com/syncliteio/SyncLite), low-code, comprehensive relational data consolidation platform backed by patented technology. SyncLite excels at performing real-time, transactional data replication and consolidation from a myriad of sources including edge/desktop applications using popular embedded databases (SQLite, DuckDB, Apache Derby, H2, HyperSQL), data streaming applications, IoT message brokers, traditional database systems etc. into a diverse array of databases, data warehouses, and data lakes, enabling AI and ML use-cases at all three levels: Edge, Fog and Cloud.SyncLite offers countless data integration architectures and can scale to hundreds of GBs per hour data transfer rates across data pipelines Show less

Engineering manager for Azure SQL DB/SQL Server Data Integration team that owns various topics including Change Data Capture, P2P Replication, SynapseLink, DataSync, Backup/Restore, Import/Export etc.Hired and grew a team of 20+ engineers in Microsoft IDC to own and deliver critical features and established on-call assistance team for data integration topics in IDC.

Arcion Labs (Formerly BlitzzIO)Architected from ground-up a cross-database real-time database replication engine that can scale-up, scale-out, support heterogeneity of 10+ leading database systems both as sources and targets with all sorts of source-target combinations, provide a high availability clustering solution, ensuring complete data consistency in the presence of any kinds of failures throughout the data pipeline.Bootstrapped India office, hired and led a team of… Show more Arcion Labs (Formerly BlitzzIO)Architected from ground-up a cross-database real-time database replication engine that can scale-up, scale-out, support heterogeneity of 10+ leading database systems both as sources and targets with all sorts of source-target combinations, provide a high availability clustering solution, ensuring complete data consistency in the presence of any kinds of failures throughout the data pipeline.Bootstrapped India office, hired and led a team of engineers with mixed experience levels in taking up the full product development life cycle involving architecture, design, development, demos, customer POCs, support/maintenance activities etc.Delivered over 200 major/minor product releases over the span of 2+ years ensuring agile response to customer issues including NFRs in core engine, new data pipelines, bug fixes etc.Demonstrated over 100 GBs per hour data transfer rates for real-time replication across industry leading cloud hosted databases and data warehouses. Show less

Delivered architecture, design & implementation of various features including synchronized backup-restore, system replication and table replication for dynamic tiering solution (a.k.a. extended storage solution) for the In-memory DBMS SAP HANA by extending it to a disk-based columnar DBMS Sybase IQ. The delivered solutions facilitated a seamless operational experience across petabyte-sized databases deployed at leading organizations.

Delivered acceleration of LIKE predicates using Sybase IQ TEXT indexes: Use of specialized Sybase IQ TEXT indexes for performing prefiltering ahead of execution of LIKE predicates.A performance improvement of 5X-10X was seen for a customized query workload on TPC-H dataset with this enhancement.

Delivered on query scale-out initiatives accomplished in Sybase IQ 16.0 product release: Predicate scale-out: Sybase IQ being a column-oriented DBMS is extremely optimized to execute SQL predicate on columnar data. A major challenge was to make the distributed execution of predicates faster with both scale-up and scale-out. We revamped the entire predicate execution layer with introduction of fully scalable parallelism strategies for entire predicate trees apart from parallelism… Show more Delivered on query scale-out initiatives accomplished in Sybase IQ 16.0 product release: Predicate scale-out: Sybase IQ being a column-oriented DBMS is extremely optimized to execute SQL predicate on columnar data. A major challenge was to make the distributed execution of predicates faster with both scale-up and scale-out. We revamped the entire predicate execution layer with introduction of fully scalable parallelism strategies for entire predicate trees apart from parallelism in just individual predicates which dramatically reduced the amount of data transfer and communication during the distributed predicate execution .We introduced an intelligent predicate optimization module which selects right kind of access methods and parallelism strategies (by reordering predicates in the tree if necessary) for executing predicates taking into account the overall cost of work distribution. A significant scale-out factor of 0.9 was achieved in Sybase IQ predicate execution layer (measured on query workloads over TPC-H dataset)Enhanced Sub-query flattening to help scale-out.Subquery flattening (or subquery decorrelation) can be often achieved with introduction of a join operator and a grouping operator above it to eliminate spurious duplicates coming out of the join. We applied a number of optimizations like eager grouping, reusing existing projections for grouping process etc. to eliminate/reduce the size of operators in flattened query plans. Reduction in size of operators directly helps in improving scale-out as less data have to be shared across nodes during distributed query execution.Improved 5 TPC-H queries with significant performance enhancement in Sybase IQ for both scale-up and scale-out scenarios with this effort. Show less

Developer on VMware - Capacity Planner project. I was part of the server consolidation component team .I worked on developing a number of new features in the application as well as dealt with production database issues, query performance tuning etc.Improved data ingestion performance by orders of magnitudes in procedural SQL scripts by converting row-oriented execution into batched executions.