Engineering Lead, Data
Current•Promotion path at Epic is Data Engineer->Senior Data Engineer->Engineering Lead, Data•Manage a team of Engineers responsible for building and supporting the Data Warehousing environment at Epic•Own the execution of a project to migrate from Hive->Databricks-- Previous experience highlights at Epic --•Own, design, develop, and maintain a 1.5 PB Snowflake data warehouse adopted by over500 users for ad hoc queries, BI analysis with Tableau, and custom application integration•Develop Python process that loads 500-700 million records every 10 minutes into Snowflakefrom Amazon S3 for Fortnite and other products•Design Python process that determines data types of JSON keys and automatically parsesthem into their own tables with properly typed columns•Train users to create and tune Tableau workbooks for live query capabilities against a near-realtime data pipeline in Snowflake•Develop batch ETLs in a Hadoop environment using Apache Pig and HiveQL•Support a 90 PB Hive data warehouse using a batch Hadoop environment built on AmazonEMR, EC2, and S3•Develop Python processes to scrape websites and call third-party API endpoints forintegration into a Hive data warehouse•Mentor Analysts and Engineers for process and ad-hoc query development in Snowflake,Hadoop, and Tableau workbook creation