Software Engineer
Current• Owned, designed and built Spark compatible SDK in R. Helped internal stakeholders and customers (datascientists, analysts, and clinical informaticists) migrate from JupyterHub (base R and Python) to Azure Synapse(Spark). Spark in Synapse notebooks offers major time and cost savings both internally and for our customers.• Built various SDK enhancements centered around stakeholder/user feedback, bug fixes, and Spark optimization.• Owned feature extraction service for COVID dashboards. Optimized Pyspark queries to reduce Spark operations by50%. Created custom features to handle more complex logic. Migrated service through multiple schema changes.• Drove initiative to migrate feature extraction service into Azure Synpase notebooks. Managed stakeholdercommunication and expectations, managed cross-team work and personally handled all of the associatedengineering work. Drove costs down by 75%, addressed technical debt by making processes more simple tounderstand and debug, demonstrated what an end-to-end workflow looks like in Truveta Studio with large data andspark operations.