• Implemented and designed anomaly detection system from raw data ingestion to analysis resultusing Scala and Spark.• Optimized level of parallelism for Spark jobs to reduce 50% runtime and improve memoryallocation.• Implemented to be easily extensible for feature updates or addition of signal models.Algorithm Design• Developed machine learning component that monitors for change in selected KPI’s. Resistant to outliers and handles missing data.• Developed… Show more • Implemented and designed anomaly detection system from raw data ingestion to analysis resultusing Scala and Spark.• Optimized level of parallelism for Spark jobs to reduce 50% runtime and improve memoryallocation.• Implemented to be easily extensible for feature updates or addition of signal models.Algorithm Design• Developed machine learning component that monitors for change in selected KPI’s. Resistant to outliers and handles missing data.• Developed Spark/Scala code with UDFs, DataFrames, and RDD for data aggregation andexploratory statistical analysis.• Developed efficient algorithm to analyze data in tar.gz files using Spark API over Hadoop Yarnwith HDFS data. Show less

Developed a Bayesian Online Changepoint Detector to determine changes in data from different probabilistic distributions using Scala. Will determine if each incoming sample belongs in the same distribution based on how well the data fits the believed distribution. Case classes are used for each probabilistic hypothesis to enable easy plug and play. Entire program is almost entirely immutable, as is common in functional programming, for parallelization using Spark. The program fits in with Spark… Show more Developed a Bayesian Online Changepoint Detector to determine changes in data from different probabilistic distributions using Scala. Will determine if each incoming sample belongs in the same distribution based on how well the data fits the believed distribution. Case classes are used for each probabilistic hypothesis to enable easy plug and play. Entire program is almost entirely immutable, as is common in functional programming, for parallelization using Spark. The program fits in with Spark Cluster Framework for large scale regression analysis on input parameter performance. Performed Data Visualization with D3.js using DC.js and Crossfilter.js. The visualization is in-browser OLAP (Online Analytical Processing)-like data aggregation with real time filtering through web-interface manipulation. Developed program for Data Analytics using Python Pandas to analyze aggregated data from network base stations, typically hundreds of thousands of rows in a Dataframe for each KPI. Show less

Collaborated in developing an internal use data and milestone management platform using Javascript and MongoDB Database.

Implemented a simulator in C++/ROOT to investigate hardware response of millions of electrons through a high voltage potential upon Silicon Photo Multiplier planes. A GUI is used for experimental set-up.