Working on the Azure SQL Database Hyperscale

Working on the design and implementation of metadata systems used by the Azure Data Lake/Cosmos store team.

Working on the design and implementation of metadata systems used by the Azure Data Lake/Cosmos store team.

- I was part of the core group that designed the scalable in-memory OLTP engine in SQL14 and SQL15. As part of this we designed parallel checkpoint, parallel recovery, and parallel merge to keep storage in check.- I ensured that bootstrap worked from last checkpoint and log tail. I was responsible for integrating the in-memory engine with the traditional engine seamlessly which avoided app rewrite to use the new engine. I designed and implemented high availability for in-memory engine which include continuous redo on secondary, independent checkpoint and failover both with and without data loss.- I implemented several native (C++) coding tasks full of lock-free mechanisms and other advanced synchronization techniques. My attention to details, code health and code documentation resulted in improved code reliability in my team.- I pioneered new way to write and organize native unit testing. I implemented complex long haul with log shipping stress tests. The test code is written in C++ and exercise the product code within the same process. The tests I authored were referenced in the exercising and stressing of complicated scenarios like failovers and storage stress scenarios. This cut down test investment significantly for my team.- I designed and implemented a test mechanism to reliably simulate a cluster of database replicas into a single process. That allowed developers to easily launch one single executable which contain test and product code that exercises complex cluster scenarios like: continuous redo on secondary replicas; failover; failover with data loss; independent checkpoint; etc. This level of testability was unprecedented in SQL Server.

- (SQL AlwaysOn) I designed and implemented the connectivity WSFC (Windows Server Failover Cluster) integration. Users can create the “name” resource (or client access point) either by using T-SQL or by configuring WSFC.- I implemented appropriate supportability hooks and unit tests that did not require cluster setup.- MatrixDB (or transparent scale out): I designed and implemented elastic scale where one can add or remove nodes transparently. As part of this I implemented versioned in-memory topology and appropriate debugging and supportability hooks. I implemented a new WMI dll which hosted 2 WMI classes used to query the topology by non T-SQL clients like the SQL Server Setup, powershell scripts used by tests and other C# automated tests.- Componentization and testability: Given my passion for code health, I prototyped, proposed and then worked with a team to implement a multi-binary engine where a subset of DLLs can be re-used by itself for testing and other needs. Monolithic SQL server engine was not only hard to test but was hurting productivity.- I drove the adoption of TDD methodologies in SQL engine resulting in much better unit testing and higher quality. The cost of testing concurrency issues in distributed or multi- process systems reduced dramatically with much better outcome. I mentored and guided junior developers on best engineering practice, in-memory technologies and high availability.

I worked in Performance Point Server 2007 and O14 and I was responsible for the data movement and data loading features. The work involved multi-threading, queue, T-SQL statements generation.

- I was responsible for the architecture and implementation of the company main products including BI, data variable printing and OMR.- I implemented CVS for source control and NUNIT to help quality assurance. We used to have 500+ tests that must pass before any version was released- I implemented financial controls and projects controls (features, bugs, tasks, etc) using Kankei DBM- I lead a team of five developers.

Responsible for the Brazilian team in an international project for a multinational food company. It was a business inteligence (data analysis) system that were successfuly deployed.

Responsible by the architecture of products and consulting developed by Vesta in products like EasyPay(payment using Internet), VestaMarket (B2B), Comprasnet (Auctions), eGov (Government B2B) and thekernel library VCCE (used in consulting like Comprasnet).

I grew up to become the main developer and build technologies that lasted for 10+ years.