Data Engineer
Current→ Create and deploy ETL solutions to transform transactional data into dimensional models – transactions done with SQL, but executed with Python (Lambda). Built overhead architecture for tracking and error handling.→ Pick up daily data drops into our data lake and process / load the data from S3. Maintain partitioned history table in Redshift Spectrum for error handling and monitoring.→ Implemented Python scripts to invoke Alteryx workflows stored on Alteryx Server, executed with Lambda and orchestrated by Step Functions.→ Set up, optimized, and maintain Redshift clusters in dev. and production. Scheduling maintenance on key tables to maintain reporting speed.→ Managed data migration from on-premise data centers to AWS – data exports include daily truncate / load and full load plus CDC.→ AWS administrator for team and manage permissions company-wide.
