Backend Software Developer
CurrentIBM DB2 Columnar Database Engine (C++): •Developing new and improving existing features of the DB2 relational database, more specifically, the Columnar Data Engine in DB2 for Database and Date Warehouse, which is efficient for enterprise large queries and data analytics, etc.•Focusing on the data compression and storage of the Columnar Data Engine. Working on not only saving storage space, but also improving the performance from technics like column dictionary and page dictionary.WatsonX.data (IBM’s data platform for Gen AI) and LakeHouse works:Prestissimo (Presto C++ worker):•Developing code for Prestissimo query engine, which is to improve the performance of Presto with C++ implemented Velox worker. Velox is an open-source C++ native acceleration library designed to be composable across compute engines. •Implementing code of scalar functions and writing gtests to test it. •Integrate Prestissimo and its dependencies in Docker-compose (Hive, MinIO, JMeter, etc) for the first time, for everyone to demo and testing, which is helpful for the whole community. •Write performance benchmark tests (Parquet reader) and put components together to adapt well-tested IBM’s workload to test the Presto/Prestissimo engines. Debug and fix problems.Optimizer Plus:•Use DB2 enterprise-proven query compilation technology coupled with advanced query rewrite and cost-based optimization techniques to optimize the query performance of Presto/Prestissimo.•Working on adaptation of the difference between DB2 and Presto queries to improve the better performance. Solving other system compatibility issues and integrating the component with WastonX.data.•Managing the code base on git.•The combination of Presto C++ and Optimizer Plus was able to deliver better price performance compared to Databrick’s Photon engine, with equal query runtime at less than 60% of the cost, derived from public 100 TB TPC-DS Query benchmarks (tested internally by IBM).
