Radically simplify DataOps

Employee #3, and part of the founding team of Unravel Data, I developed the first version of many features in Unravel. My team leverages AI to develop sophisticated application performance management and optimization for big data applications such as Spark, Databricks, EMR, Redshift, Impala, Hive, Tez, LLAP, MapReduce, Athena, etc.

Software engineer building infrastructure for managing big and real-time data. Specialized in Hadoop, Hive, Hue.

Major Projects:- Database sharding in SQL Azure (version 1) – support scaling out the database tier to hundreds of SQL Azure databases- Management service for SQL Azure (version 1) – support GUI and programmatic access to manage SQL Azure databases - Support for SQL Azure database firewallMajor Accomplishments:- Authored functional design for user scenarios, the client-tier, and the mid-tier of the management service- Authored company-wide engineering sign-off requirements such as security and supportability- Planned, prioritized, and scoped customer scenarios- Drove project management with teams across US and India to complete projects on time under tight resource and time constraints- Designed information architecture and user interface for the management portal, designed user studies with UX Labs- Engaged with external customers, including live and video presentations

My research in the PhD program of the University of Wisconsin-Madison focused on issues that arise in managing sparse relational data sets. A motivation for this work is to allow extracting and processing structured data from unstructured data in an incremental fashion, and to allow users to query the structured data that is currently available. I considered using a RDBMS as the basis of a workbench that provides basic operations that can be combined to process data in a “pay as you go” fashion, and a wide table to store the resulting sparse data set. We conducted a case study as a proof of concept, and concluded that while the relational workbench approach is promising, its success relies on good relational database support for sparse data. Unfortunately, most relational database systems are not good at handling sparse data sets. The rest of my research addressed the challenges of storing, querying, and evaluating queries over an evolving sparse data set in a RDBMS. The querying problem led to the broader problem of building structured queries over a relational database system without knowing SQL and the database schema. For this problem, I proposed to combine keyword search and query forms to try to provide the best of both. I presented a systematic approach to generate forms, developed techniques to handle keyword queries having a mix of database constants and terms that appear on the forms, and considered grouping similar forms returned by keyword search to help users more easily find the right form. Our real-life user study showed that our keyword search implementations are efficient and generally effective, and that grouping forms is an important post-processing step of keyword search to lead users to the right forms.See publications for more information.

Developed a C# prototype for database workload analysis with query plan visualization. The prototype allowed users to view anomalies in workload performance, identify where the anomalies occur in the query plan, and drill down to see the correlated counters.

Developed techniques and extended existing code in Microsoft SQL Server Tuning Advisor to exploit the ordering of SQL queries in a workload to improve the performance of that workload over previous approaches that considered a workload as a set of queries. The work was published in SIGMOD 2006.Patent No. 7,664,774 – Sequence-based Physical Design Tuning

Extended the data model of the Java Content Repository (JCR) and developed XML type support to store XML documents directly - previously, JCR only supported XML storage by shredding XML documents into other primitive property types such as String and Binary.