Hadoop Developer
CurrentHudson DataTPC-H Benchmark Testing (Progressive Insurance)Description: Completed benchmarking of Pivotal HDB Hawq, Apache Hive, Apache Spark, and Actian Vector-H for presentation to Progressive Insurance to determine technology to be used in future in-house Big Data solutions. The benchmark was completed using Transaction Processing Performance Council (TPC) H benchmark for database generation and standardized queries on the generated data. Data sizes tested were intervaled between 1 – 500GB. Hadoop Distribution used was Hortonworks Data Platform 2.4. Main Role: Researched Pivotal HDB Hawq and installed it on a cluster using Hortonworks Data Platform 2.4. Wrote a Python program to deploy the test, record the results, and format the end-results with calculated statistics for input to Tableau.Technologies: Python, Hawq, PXF, Hortonworks HDP 2.4, Hive 1.2, Spark 2.1, Tableau, Centos 6.5, Actian Vector-H Clear Analytics Hive AdapterDescription: Clear Analytics is an excel-based query designer which works as an adapter to connect various data sources to add Business Intelligence while maintaining secure access. The project was to add Hive as a datasource for the Clear analytics tool, allowing the Hive Metastore to be queried as a datasource.Main Role: Developer on the team analyzing and upgrading and enhancing the existing C# codebase to add the feature of Hive capabilities. Technologies: Hive 1.2.1, C#, Clear Analytics, JDBC