Data Engineer
Current• Developed Python code for processing images based on client’s requirements using the OpenCV-Python library and automated the pipeline for processing of images residing in one AWS S3 bucket and upload the processed images to a new S3 bucket.• Implemented SQL queries to obtain a summary of the pricing history of client’s product and use the information to create a dashboard using the Python Dash library and Tableau. Twitter Sentiment Analysis:◦ Demonstrated a machine learning pipeline where:▪ Live tweets were extracted using Twitter API and processed using AWS Kinesis and stored in an S3 bucket.▪ Exploratory Data Analysis (EDA) and Sentiment Analysis of the live tweets was performed on a Databricks platform using the Pyspark Machine learning library.▪ The results of are stored in S3 bucket, tweet analysis data tables were created on Amazon Athena and tweet analysis dashboards were created using AWS QuickSight. Back Order Prediction:◦ Designed and developed Python code to perform the EDA, cleaning and machine learning model to predict whether anautomotive parts item was back ordered using Classification algorithms.◦ Acheived 10% increase in prediction accuracy score of new back orders on a new automotive manufacturing dataset using Gradient Boosting methods. Web-Scraping:◦ Implemented a Python script to scrape a Realtor’s website using Beautiful Soup and Selenium libraries.◦ Developed code in Pandas Python library to perform EDA on the scraped data and used the cleaned data to develop dashboards that will guide a new home-buyer choose the area and a home of their choice. Relational Database/ SQL Query:◦ Implemented data scraping from AirBnb website, data cleaning using Pandas and stored the data in MySQL database tables.◦ Defined business objectives and queried the data required to accomplish the objectives using SQL Workbench. SQL Queries were implemented with Common Table Expressions and window functions for improving readability and performance.
