Data Engineer
Wrote MySQL queries to validate manually inputted data• Able to find missing, invalid, or suspicious columns in tables with thousands of entries• Implemented as model queries in Laravel for easy use as endpoints• Fully tested for accuracy and functionality with suite of unit testsDesigned and implemented Coordinate Country Verifier to find misplaced coordinates• Can find errors in manually inputted coordinates (e.g. if coordinates supposed to be in Canada but actually placed in middle of the ocean)• Found suitable GIS country boundary geometry and created program to load into MySQL database• Created Laravel + MySQL program to comb through 4000+ database entries and fetch those with mistaken coordinatesCreated AWS S3 pipeline to split PDF documents into page images• Utilized Python PyMuPDF library• Deployed on AWS Lambda for easy multiprocessing to handle 4 million documentsPorted Logo Scraper into Laravel backend for use as API endpoint• Eliminated network issues and costs associated with relying on AWS Lambda