Experienced Software Engineer with over 10 years in the Information Technology industry, specializing in Hadoop/Spark development using Big Data technologies such as Hadoop Ecosystem, Spark Ecosystem, PY Spark, and extensive ETL development experience. Proficient in tools including HDFS, Spark, Airflow, Yarn, MapReduce, Kafka, Hive, OOZIE, HBase, Impala, and Apache Druid. Skilled in containerization with Docker and Kubernetes, data engineering, pipeline design, and deployment. Successfully implemented data transformation solutions using Python and Databricks, improving data accuracy and efficiency for healthcare and financial clients. Strong in data governance, profiling, and cleansing techniques, and experienced in Hadoop framework layers, data integration tools like TALEND and Apache NIFI, and Pandas/NumPy for data visualization. Knowledgeable in Snowflake's architecture, Data Lake design, and proficient in PYSpark SQL. Collaborative with DBAs and developers, experienced in working with CI/CD pipelines, source code management tools, and Hive data warehousing. Also experienced in real-time data streaming with Kafka and Flink-based data processing pipelines.