Associate Software Engineer
CurrentRoles and Responsibilities: • Proficiently designed and developed solutions for Big Data using Hadoop ecosystem technologies: HDFS, Hive, Sqoop, Map Reduce, and Apache Spark.• Played a key role in Hadoop migration projects and Proof-of-Concept (POC) initiatives.• Demonstrated expertise in Apache Spark on GCP, leveraging DATAPROC, Big Query, and AIRFLOW, with skills in Scala and Python.• Acquired a deep understanding of Hadoop architecture, including HDFS, Job Tracker, Task Tracker, Name Node, Data Node, and the Map Reduce programming paradigm.• Installed, configured, and effectively used Hadoop ecosystem components, encompassing Hadoop, Map Reduce, HDFS, Hive, Sqoop, Pig, Zookeeper, and Flume.• Executed Apache Hadoop Map Reduce programming, crafted PIG Scripts, and developed Distributed Applications within the HDFS environment.• Displayed proficiency in Hadoop Cluster architecture and adeptly monitored cluster performance.• Managed and reviewed Hadoop log files to ensure system health and performance optimization.• Successfully imported and exported data between HDFS and Relational Database Systems, utilizing Sqoop for seamless data transfers.
