S C Email & Phone Number

加拿大 安大略省 多伦多

Toronto, Ontario, Canada

•Lead a team Migrate the 100 reconciliation jobs(Java to SQl) from Mysql to tidb using recon-ui(platform), result ing in an annual cos savings of $1500000 and an increase in performance of 70%.•Co-led to design and implement a real-time data pipeline to process semi-structed data by integrating 1B raw records from 15 different business using Flink and Kafka which result in false alarm rate from 50% to 0.01%.•Co-led to build and implement a Streamer data pipeline to process history data by integrating 200B raw records from 30 different business using Spark as backup for real-time job.•Ingested data from disparate data sources(mysql,hive) by Java API using combination of Sql, build the data-pipeline using Spark, and create data views to be used in BI tools like Datawind and Grafana.Communicate with project managers and analysts about data pipeline that frove efficiency Okrs up by 40

Guangzhou, Guangdong, China

C, Bigdata&Application July 2019 - June 2021•Process 20 business module base on Scala,building the data analysis and statistics development system based on lambda architecture•Designed an automated ETL process across billions of rows of data base on spark in python which reduced manual workload by 30% monthly.•Worked with data analyst team and process OLAP analysis function and translate those business needs into actionable reports in Tableau, saving 10 hours of manual work each week.•Deal of the basic problems in the running process of the distributed platform system(Hadoop), and optimize the task efficiency of scheduling platform(xxljob,airflows).

China

Toronto, Canada Area

• Stream services built using Java and Scala can handle 10000 users in real time• Improve performance of existing ETL processes and SQL queries to get weekly CRM summary data• Leader migrates legacy data warehouse from on-premises to AWS and Java / Spark IUM• Developed an infrastructure that processes 10 TB per day, resulting in an 5% increase in online sales for the Ad-tech division REM

Developed algorithm to convert insurance-orientated ICD-9 codes to clinical practice meaningful disease classification.Coordinated with doctors for data cleaning and variables selection to ensure data clinical meaningful and analyzable.Conducted data analysis using logistical model, LDA, QDA, KNN, tree classification and random forest method to identify high readmission risk patient and improved the accuracy (C-scores) by 30 percent.Two papers related to Readmission Project submitted and ready for review.