-Managed and maintained Cloudera Data Platform (CDP) clusters across analytics, DR, and production environments, ensuring high availability and optimal performance.-Monitored and optimized CDP cluster performance using Cloudera Manager, identifying bottlenecks and improving resource utilization.-Conducted health checks and troubleshooting of services such as YARN, HDFS, Spark, Zeppelin, YARN Queue Manager, sqoop, Zookeeper etc. to ensure smooth operations in CDP environments.-Configured dynamic resource allocation using YARN queues for better resource management and improved cluster performance.-Collaborated with cross-functional teams to implement effective data governance policies within the CDP ecosystem, ensuring compliance and data integrity.-Handled performance tuning of Spark jobs in the CDP environment, optimizing job execution time and improving resource efficiency.-Automated Spark job execution workflows using shell scripts, scheduled with crontab to manage job scheduling and resource allocation efficiently within the CDP environment.-Created and optimized shell scripts for automated backup and restore operations, scheduled with crontab to ensure timely backups of critical HDFS data within the CDP platform.-Worked closely with data scientists, data engineers and analysts to understand their requirements and optimize the data processing environment.-Created and maintained comprehensive documentation for cluster configurations, troubleshooting procedures, and support processes to ensure streamlined operations and knowledge sharing across teams.

- Deployed test, development & production-grade Cloudera clusters shadowing the Cloudera architect in an AWS environment.- Configured Cloudera clusters with core services like Kafka, Spark, Yarn, Hdfs, Hive, Hue, Zookeeper, etc.- Performed kernel tuning for CentOS along with the Linux team.- Currently working in a high availability environment.- Created two-way trust using Kerberos and active directory on Cloudera enterprise.- Used sentry for authorization in multiple projects.- Expertise in optimizing and tuning big data platforms, performance tuning & benchmarking of the Hadoop cluster.- Performed a major upgradation in the production environment from CDH 5.16 to CDH 6.2, resulting in enhanced performance and stability.- Worked on POC to migrate from the CDH cluster to the CDP cluster.- Performed POC on AWS EMR shadowing Cloudera Architect.- Worked on incident management, service request management & change management tickets. Following ITIL guidelines.- Analyzed and resolved P1 and P2 tickets with urgency, maintaining a 95% incident resolution rate within SLA.- Implemented YARN optimization techniques, resulting in a 30% improvement in cluster resource utilization.- Conducted performance tuning of Spark jobs, reducing execution time by 25% and improving overall system efficiency.- Identified and addressed bottlenecks in data processing workflows, resulting in a 40% increase in data ingestion speed.- Collaborated with cross-functional teams to define and implement data governance policies and data lifecycle management processes.- Led capacity planning initiatives and optimized resource allocation, resulting in improved cost efficiency and cluster performance.- Conducted regular cluster health checks and implemented proactive monitoring.- Acted as a Hadoop administrator, managing and maintaining Hadoop clusters, including installation, configuration, and troubleshooting.

- Monitored Cloudera cluster components including HDFS, YARN, Kafka, Hive, and Spark jobs.- Managed and resolved P4 and P5 tickets within the agreed SLA, ensuring minimal disruption to operations.- Collaborated with cross-functional teams to identify and resolve memory-related issues, reducing system downtime by 20%.- Developed monitoring dashboards and alerts to proactively identify and mitigate performance bottlenecks.- Conducted regular cluster health checks, resulting in a 15% reduction in incidents and improved system reliability.- Assisted in capacity planning and resource allocation, leading to optimal utilization of cluster resources.- Expertise in commissioning & decommissioning of nodes, creation of snapshots, and backup & disaster recovery.- Worked on AWS cloud services such as VPC, amazon EC2, amazon S3, IAM, etc.- Expertise in setting up & configuring AWS virtual private cloud (VPC) components including subnets, route tables, internet gateway, NAT gateway, security group, and EC2 instances, etc.- Implemented enterprise storage for fault tolerance, backup, etc.- Actively participated in knowledge-sharing sessions and mentored junior team members.