Data Engineer
• Planned, designed, and deployed relational database and schema architecture using Snowflake. Migrated data stored in Excel, SAP, and Dodge Data & Analytics warehouse to Snowflake.• Developed Python algorithm for creation of tables, queries, and stored procedures. Used algorithm across multiple projects to reduce scripting hours.• Automated ETL/ELT processes using DigDag reducing manual effort, optimizing procedures, and logging errors.• Set up 160 Salesforce Marketing Cloud connections using Treasure Data platform, reduced 160 table cleansing and staging procedures to 7 using Python to create and standardize schemas and tables.• Designed and implemented ETL pipelines for processing large datasets, resulting in a 50% reduction in processing time and increasing data availability by 75%.• Migrated data from AWS S3 buckets to Snowflake, designed and implementeddata pipelines to generate analytics for the ConTech team.• Used Spark to connect to web APIs. Created and modified existing storedprocedures using Spark SQL, and analyzed data to create a new taxonomy for the ConTech group.