Data Engineer
Mumbai, Maharashtra, India
Majorly worked on alternative data and geospatial data using various tools and technologies such as AWS S3, AWS RDS, AWS ECR, AWS Lambda Function, AWS Cloud Watch, Cron Jobs, MongoDB, PostgreSQL, Selenium, Beautiful Soup, APIs, python multiprocessing etc. for the day to day Data Engineering work.Created an end-to-end automated pipeline for an Indian Government Organization from the scratch.Implemented geo-hash, MongoDB geo-queries, log rotate for MongoDB logs and partitioning the data… Show more Majorly worked on alternative data and geospatial data using various tools and technologies such as AWS S3, AWS RDS, AWS ECR, AWS Lambda Function, AWS Cloud Watch, Cron Jobs, MongoDB, PostgreSQL, Selenium, Beautiful Soup, APIs, python multiprocessing etc. for the day to day Data Engineering work.Created an end-to-end automated pipeline for an Indian Government Organization from the scratch.Implemented geo-hash, MongoDB geo-queries, log rotate for MongoDB logs and partitioning the data on various levels based on the type of data.Worked on all types of data like structured, semi-structed and unstructured.Created an automated script to test all the DMS(Data Management System - Internal Data Warehouse) APIs using python decorators.Created/updated a middleware-cli and local python package so that users can use the DMS APIs easily and effectively.Helped to create an impact across the top public listed enterprises in India in BFSI,FMCG & CPG and it played a key role in enabling financial inclusion across the country Show less