Data Engineer Iii
Current- In a leading pharmaceutical company, I worked as a Data Engineer within the Artificial Intelligence and Analytics (AIA) department of the development branch. The AIA department is tasked with designing and implementing.
- Data Management: Managed both structured (SDTM) and unstructured data (doctor's notes), performing transformations, filtering, imputations, and aggregations to prepare data for model ingestion.
- Refactoring and Modernizing Efforts of Structured Data Pipeline: Transitioned the data processing pipeline to a Glue/Lambda/Step Functions job combo with a CI/CD pipeline, storing code and Terraform Infrastructure as.
- Creation of Unstructured Data Pipeline: Key collaborator in creation of OCR/Textract pipeline, initially in AWS and later transitioned to Azure. Specific responsibilities include leading development of Terraform and.
- Validation and Quality Assurance: Created numerous validation scripts and functions to ensure data integrity and accuracy, prior to, during, and after processing. contributing to the successful completion of product POC.
- Digitization Project: Developed a functional pipeline using AWS Textract for OCR, later transitioning to Azure. Created a comprehensive instruction manual on building a Terraform pipeline.