As a Big Data Engineer, my role in the Telenor AEP Project involved working with the Apache Hadoop ecosystem on the Cloudera platform. Key Responsibilities include: 1. Working extensively with various components of the Apache Hadoop ecosystem, including Hadoop Distributed File System (HDFS),YARN, Hive, and Spark.2. Utilizing QlikSense, a powerful data visualization and analytics tool, to extract insights and performance metrics from the data.3. Using MySQL to perform data summarization, querying, and in-depth analysis, enabling better decision-making based on the data insights.4. Processing and Analyzing the data provided by Telenor using Vertica, a high-performance columnar data warehouse, ensuring efficient data operations.5. Running SQL scripts to perform data analytics, extracting valuable information, and monitoring the workflow of data throughout the system.6. Creating Cron jobs and Bash scripts to automate tasks and streamline processes on the Teradata SLJM platform.7. Responsible for diagnosing and resolving any issues related to Apache Spark and Hive jobs, ensuring smooth data processing and analysis.8. Monitoring and troubleshooting the Edge Nodes (CDH, CDS) deployed on the Hadoop cluster, ensuring their optimal performance and reliability..