•Used languages/frameworks including Scala, Java, Python, SQL, Terraform, Jenkins. Evaluated options for spark including pyspark, java, and Scala. Selected Scala due to in house experience and easy transference of standard Scala concepts/syntax to Spark.• Worked with distributed systems handling 10s of thousands of messages per second. Worked to develop the existing system while keeping an eye toward modern technology like Kafka.• Worked with data at near petabyte scale. Used Kafka and Spark for data processing.• Designed & Implemented a base CICD pipeline with Jenkins/Terraform/Docker on GCP. Works for mixed Windows/Linux infrastructure. Evaluated choices for builds carefully before selecting scripted jenkins docker builds over hosted docker builds due to reliance on Windows and an uncertain choice of future docker repository provider.• Worked with Elastic Stack & Spark connectors for Elastic Search to assist in monitoring efforts and Elastic Search data loads. Experience setting up Elastic Search from scratch. Knowledge and experience with TIG stack and pros/cons.• Organized Hackathons. Topics were Logging & Monitoring/BigQuery & Data Visualization• Built Hadoop based jobs in spark & map reduce. Jobs include streaming and batch data pipelines as well as some basic behavior modeling. Spark 2 selected due to ability to share 90% of code across batch/streaming loads. Evaluated Apache Beam as alternative, determined not mature enough plus other issues.

• Worked on C# Rest API that allows customers to interact with catalog data, predictive model data, and create self-service reports.