• IT professional with nearly a decade of experience as a Senior Data Engineer, specializing in designing data-intensive applications across a spectrum of technologies including the Hadoop Ecosystem, Big Data Analytics, Data Warehousing/Data Mart, Cloud Data Engineering, and Data Visualization. • Possesses an in-depth understanding of Hadoop architecture and its integral components, encompassing YARN, HDFS, Name Node, Data Node, Job Tracker, Application Master, Resource Manager, Task Tracker, and the MapReduce programming paradigm. • Demonstrates strong proficiency in developing enterprise-level solutions leveraging Hadoop, harnessing key components such as Apache Spark, Airflow, MapReduce, HDFS, Sqoop, PIG, Hive, HBase, Flume, NiFi, Kafka, Zookeeper, and YARN. • Extensive hands-on experience in developing Spark applications utilizing tools such as Spark Core, Spark MLlib, Spark Streaming, and RDD transformations. • Proficient in data cleansing and analysis using HiveQL, Pig Latin, and custom MapReduce programs in Python. • Skilled in importing streaming data into HDFS using Flume sources, sinks, and interceptors. • Experienced in utilizing Oozie as a workflow scheduler to manage Hadoop jobs with a Directed Acyclic Graph (DAG) structure. • Familiar with common operators in Airflow, including Python Operator, Bash Operator, and Google Cloud Storage operators. • Expertise in data import and export using Sqoop, facilitating seamless data movement between Hadoop Distributed File System (HDFS) and Relational Database Systems (RDBMS) such as Teradata. • Skilled in working with various database platforms, including both NoSQL and RDBMS tools such as MySQL, Oracle, SQL Server, PostgreSQL, DB2, DynamoDB, MongoDB, HBase, Cassandra, and Cosmos DB. • Extensive experience in designing, implementing, and optimizing data models in Apache Cassandra, leveraging denormalization, wide rows, and partition keys for scalability and fault tolerance. • Proficient in designing and implementing secure data ingestion pipelines using Apache NiFi, leveraging robust security features such as user authentication, role-based access control (RBAC), and secure data transmission protocols (e.g., SSL/TLS). • Familiar with scheduling and workflow orchestration tools like Automic, Control-M, and Tivoli. • Strong understanding of Data Warehousing concepts with hands-on experience in implementing complete life cycle projects, including data modeling, OLTP & OLAP database system design using ER diagrams, ETL processing, and data marts.
