Data Scientist, Core Compliance Tech
Current- Worked with global teams on improving both efficiency and quality of Global Surveillance Solution (TotalConduct) by data science and analytics. Responsible on performing research, developing analytical tools and new features of the platform.- Led several end-to-end projects, including defining requirements, transforming requirements into technical design, implementations, code review, integrating the final product with production cluster.- Designed and implemented big data analytical pipeline using Spark algorithms to calculate multiple metrics of different surveillance rules, helped clients to understand how the surveillance rules performed, and productionized the PySpark application on the cluster.- Led and coached new joiner on project design and technical skills.- Developed many python packages for the team on data extraction and big data calculation algorithms across multiple projects.- Conducted extensive data analysis on the platform output, processed large and complex structured data (TB level).- Led an effort on creating curated dataset, built a framework with Spark, and coached a team on the skills.- PySpark, Pandas, NumPy, json, ast, collections