Performed below magic:- Develop ETL pipelines for data sources and application on-premises/cloud as per business needs.- Optimize memory and runtime for data flow by using miscellaneous approaches.- Develop, integrate and maintain microservices.- Develop custom libraries to achieve code reusability.- Create web automation applications for automation testing and web scraping.- Create and deploy event-driven serverless applications on cloud platforms. (AWS Lamda/Azure Functions)- Develop applications to scrape and crawl websites and mobile applications.- Pack applications with dependencies into Docker image and deploy on cloud platforms. (AWS Batch/Lambda)- Create and deploy RESTful APIs.- Create unit tests for automation testing.

Scraped data from websites, mobile applications, APIs and local files.Organized and structured raw data and stored into database.Developed several Windows OS based applications using Python for business to run crawler on autopilot.Analyzed, filtered big data to deliver dedicated chunk of data.Created Restful APIs to access scraped data as per business requirementsDeployed crawlers on servers to run on recurring schedule to perform end-to-end functionality for clients seeking data on a periodic basis.