Senior Systems Operations Engineer
Current• Hadoop cluster Design, Installation, Capacity planning, Configuration, fine tuning, Optimization and Administration.• Expert on HDFS, YARN (MRv1 & MRv2), Hive, HBase, Impala, Zookeeper and Hadoop Eco-systems.• Managing 500 nodes cluster for Production and various non production clusters.• YARN – MRV1 to MRV2 Migration on YuMe software platform.• Handling high severity production issues.• Cluster performance analysis and fine tuning the cluster.• Cloudera CDH Major Upgrade: CDH 4.4 to 5.1 and subsequent Minor upgrades to 5.16.1• Scaling the cluster based on the Product requirements.• Hadoop Cluster - High Availability for Name nodes to prevent single point of failures.• Fair scheduler implementation on YARN.• Ensure data distribution by running HDFS Balancer.• Hadoop cluster monitoring using various tools - Ganglia, Nagios, Hannibal and Cloudera Manager.• Hadoop Security - Kerberos implementation on Hadoop cluster for authentication and authorization.• Sentry implementation for RBAC (Role Based Access Control).• Closely working with ETL/DT Ops/Data Science and BI teams to resolve business critical issues.• Handling data ingestion issues around ETL framework.• Handling data migration between the Hadoop clusters.• Knowledge on Spark and Kafka.• Knowledge in AWS - EC2, EBS, S3, RDS.• Backing up source truth of files into S3 – Analytics and Recovery cases.• Handling Java application issues and Troubleshooting.• Build and Deployment on various environments for YuMe’s software platform.• Played key role in Tomcat Migration: Major and Minor releases on Production and non-production environments.• Participating in onsite/offshore status calls for new Implementations.• Creating SOP documents and mentoring team.• Good knowledge of Shell Scripting skills.• Exploring NoSQL databases – Mongo DB, Redis and Aerospike.• Supporting incidents, service request, and problem using JIRA.