Lead Software Engineer, Data Platform
CurrentLead Engineer responsible for setting direction and priorities of the team in the context of and in collaboration with the rest of our organization. Focused on minimizing delivering nonlinear value while minimizing time-to-value with an emphasis on reliability and scalability.Some tools I work with: Python, Spark, Scala, AWS, Docker, Terraform, Databricks, Airflow,, Parquet, GCP, telegraf, grafana, Prometheus, GitLab, R, EMR and Glue, DjangoAs an engineering leader at Nuna I work across our platform organization to make sure we're delivering the right data to the right users at the right time. Along with my team I support analysts and engineers in using the latest tools to deliver the most value to our customers.Our data platform has to support ingesting, cleaning, QAing, enriching and exporting data to and from a wide range of places. A few of the tools I use and things I've done at Nuna:Developed an API to execute and serve generic financial calculations for a web frontend and determine payment levels for multi-million dollar contracts.I led the design and development of a runtime configuration system for flexible, generic product data pipelinesDesigned and built the foundation of a notebook-based analytic workflow for analysts and data scientists to write production code.Mentored Data Scientists on engineering best practices and helped them deliver production-quality Spark and ScalaI've also:I maintain and write new features for our data quality tooling (Spark/Scala)I identified a persistent issue with our build process and developed and implemented a testing strategy to eliminate an entire category of CI failures (Jenkins, Harbormaster).I build and maintain services on AWS (EC2, S3, SQS, EMR, RDS, CloudFormation, IAM)I've been on the founding engineering team for a new product, helping to define the data models and processes that will power the future of the company (Protocol Buffers, Scala).