1. ETL and data engineering• Create python scripts to extract data from sources such as MS SQL Server, MySQL, Oracle, Spark, Hive and Teradata.• Develop python scripts to join eight tables from various sources. Design composite primary keys for all tables by applying self-defined or standard aggregation functions. Build lookup table to correct data or fill missing data. • All tables are first joined into two subgroups by relevant primary keys and then into a single table. Reconcile data inconsistency for the same-name columns in different tables under same primary key. 2. Supervised learning• Predict the winning probability of a business opportunity. Group each high cardinality feature up to 20 categorical levels according to its distribution. Logistic regression, SVM, random forest, bagging and boosting algorithms are used. • The dimension of one-hot vector can easily pass a thousand even after feature selection, which increases the model variance. To improve the model, we apply target statistics and feature hashing to decrease the dimension. • This significantly increases the model performance. For the same precision required by business, the recall is increased by 50%. At very high precision the recall may be increased by more than 100%. 3. Unsupervised learning • Investigate what features contribute to a business class. Although an unsupervised problem, labels are used to select features to avoid the disruptive effect of using too many features in clustering. • K-mode and K-prototypes are employed depending on whether continuous variables are discretized. Well defined clusters are obtained as shown in a nonlinearly mapped t-SNE plot. 4. Visualization • Create visualization package for the team. Develop modules to calculate all metrics of classification models for visualization. • Develop functions for plotting interactive 3D stacked bar chart to show business insights.

• Used classical time series models such as autocorrelation function, moving average, etc. to predict market trending. • Used linear and nonlinear regression machine learning algorithms, tree-based models and neural networks to calculate the entry and exit prices of financial market. • Developed an automatic stock trading system, which can trade stocks without human intervention and handle up to 50 stock orders per second. • Developed a stock symbol selection system to manage trading risk. High volatility stock symbols are selected and then filtered first by their correlation to the market index and then by the pair correlation among each other. High volatility and small correlations significantly reduce the risk of trading portfolio. • Calculated the general volatility of the portfolio and used put option for the market index to hedge short term risk arising from unexpected events such as geopolitical issues, and from the large bid/ask spread of high volatility stocks. • Designed trading socket clients and their callbacks in different threads based on Interactive Broker’s C/C++ API, which involves multi-thread programming, socket programming and intensive numerical calculations. • Develop historical data downloading client, real-time market data feeding client, which are running in separate threads to serve the main trading client. • Combine three C/C++ projects from open-source technical analysis code, Interactive Broker’s API source code, the self-designed trading system, and together with SQL server, into a fully functional trading system. • C/C++ sample project can be found at https://github.com/ljyang100/dataScience/tree/master/programming/sampleProject.

• Did research and simulations on the behavior of electrons in semiconductor quantum wells and other nanostructures. • Conducted theoretical and computational investigation of two-dimensional correlation spectroscopy applied to chemical, biological systems such as photosynthetic complex, proteins. • Research papers can be found at: https://github.com/ljyang100/dataScience/blob/master/mathematics/physics/selected%20publications.ipynb