Data Engineer
CurrentAs a data engineering consultant at Effectual I work with clients across all industries to improve their data ingestion, storage, and analytical capabilities. I perform in a variety of roles including technical lead, architect, and developer. Below are several recent projects.đź”· Chatbot Assistant đź”·Spearheaded the development of a company chatbot that leverages retrieval-augmented generated (RAG) to derive context from 1st party document corpora. The project has led to increased efficiency for several internal teams and has improved the security poster of the organization by preventing data leakage.đź”· ETL Migration to AWS đź”·Led the migration of several hundred data ingestion pipelines from TIBCO iWay Service Manager to AWS serverless technologies for a large financial institution. In my day to day I am responsible for gathering requirements, designing replacement solutions, writing the infrastructure-as-code (IaC) and source code, and pair programming with the clients developers to teach them about the many new technologies they will be responsible for maintaining.đź”· Snowflake ELT to Open Source ETL Cost Saver đź”·Developed and packaged a solution to save clients money by moving them from doing extract-load-transform (ELT) on Snowflake to extract-transform-load with PySpark on AWS Glue. This solution helped clients realize cost savings of 20-40% on data ingestion and curation costs which has enabled them to invest that capital back into their business to further grow capabilities.