BigData system engineer - Hadoop & Kubernetes

Bigdata system engineer- Hadoop & infrastructure

Automotive industryProject Description - Responsibilities:• Provide 2nd level infrastructure support for multiple hadoop clusters (Hortonworks)• Manage HDFS based on the requirements• Cluster maintenance, commissioning and decommissioning Data nodes, cluster Monitoring, troubleshooting, managing & revieweving Hadoop log files.• Monitoring and supporting systems (SLES 11) and hadoop services• Managing and reviewing Hadoop Log files as a part of administration for troubleshooting purposes.• Analyzing system failures, identifying root causes, and recommended course of actions.• Documenting the systems processes and procedures for future references.• Working with systems engineering team to plan and deploy new hadoop configurations• Monitoring multiple hadoop clusters environments using Nagios.• Automation of daily and reocurring tasks (using scVenus and shell scripts)• Kafka server administration - Topic and user management

Resolving daily business tasks.(OS tasks, performance, design, network/HW issues). Bash scripting to improve the performance of the team. Acting promptly & reliable to ensure customer's satisfaction. Cooperating with international teams/colleagues (Germany, UK, Czech rep.) daily.