Data Engineer
Current- Built RestAPIs that outputs the current status of data migration. - Setup EC2 instance that runs an ETL pipeline. - Data cleaning of phone numbers. - Unit test cases that verified the Ticket Creation API - Build an API workflow in Python that migrates data from one CRM application to another. Deployed it on AWS EC2. Automated data validation, error handling and logging processes to ensure data integrity. - Setup spark jobs to migrate full data load of data size of 2TB.- Built deltalakes on AWS S3.