Senior Software Engineer
Current Learned Hadoop/Hbase basics. Took over ownership of managing two remote production Hadoop clusters (CDH3), and multiple staging/test clusters using Cloudera Manager. Setup a new cluster to use CDH4, added HA for Namenode and JobTracker. Created a new production hadoop cluster using CDH5. Added Phoenix, Flume and Hue. Wrote multiple Perl scripts to handle logging, archiving and ingestion of customer data. Updated Perl and Bash scripts to work after moving an entire setup to a new remote data center. Modified multiple Perl scripts: cleaned up, commented & added new code to find the first an available remote system from a list. Modified to use rsync via ssh for an entire directory at once instead of one file at a time. Learned a little ElasticSearch. Setup a db, wrote a Java class to add data and search by all or select indices. Delivered the Java class to production, replacing standard db for two select tables. Create a mysql db with two tables, one having a compound key. Wrote & debugged a Java lib using hibernate to interface with the tables. Wrote and debugged a Java program to interface with multiple db’s using Jedis (redis) and Spring-boot. Added junit tests, embedded redis and Swagger to test the code and REST interfaces