Perform data migration solutions using various data types, building data pipelines to store, clean, and ingest data from marketplace to warehouses within Hadoop and AWS cluster environment for the digital advertising space. Specifically, we provide solutions handling Big Data within AWS EMR utilizing Python, Hive, Spark, while worked with tools like PostgreSQL, Kafka, AWS S3, Glue, Redshift, Athena, RDS.

Project utilizes containerization applications to contain research artifacts in order to guarantee repeatability and reproducibility of artifacts regardless of deployment issues. Utilize AWS S3 and EC2 to provide artifact environments, handle data using Hive, Pig, Python and Spark. Also tasked to automate metadata documentations based on log files and test applications on Docker environment

Collect, manage, and drive insights from data for client consulting pertaining industrial noise control. Utilize Python and SQL for data cleaning, perform ad hoc querying on Hadoop clusters in Hortonworks, and manage databases in MySQL. Generated report for presentations using Tableau