I have over 7 years of experience in the IT field and nearly 3 years in hadoop. My expertise comes from various fields including admin, architecture and development inbig data.As a hadoop admin I deploy and maintain the hadoop cluster, add and remove nodes using cluster monitor tools like Ambari, Nagios or cloudera manager I managed and reviewed hadoop log files, backup and recovery tasks, troubleshoot application errors and ensure it doesn't happen again.I was building data lakes from scratch, my data was coming from multiple sources such as RDMBS and live streaming data. I used scoop to transfer data from RDMBS to HDFS. Kafka for live streaming data. By utilizing Kafka I streamed millions of data per day. Once the data being part of HDFS I used map reduce to process and transform unstructured, semi structured data into structured format and load it to HDFS.Output data stored into table using hive and I query data accordingly.Spark is a fast and easy data processing framework. I love using spark. As an admin I built around 400 nodes cluster for processing 2 to 3 TB of data every single day. I used Nagios to setup an alert to monitor the cpu and health of the cluster. I installed Kerberos to secure the cluster in active directory for security and Vulnerabilities. On average I processed almost 20 TB of data per day As we all know spark is faster than map reduce because of in memory processing I was able to fasten the process. If in case any data was lost, RDD lineage helped me to reconstruct the lost data.I also installed Kafka on virtual machine, created topics for different users and installed zookeeper, brokers, schema registry, control center in multiple machines. I also used SSL security for different user and assigned user to multiple topics by creating users log in.