-8+ years of experience in end-to-end enterprise Data Engineering and Big Data projects.-Proficient in requirement analysis, design, development, testing, and maintenance of data management and processing systems.- Adept in ETL processes and documentation.- Expertise in designing and implementing Data Management Lifecycles, encompassing data ingestion, integration, consumption, delivery, and reporting.- Strong background in Big Data environments, including Hadoop components for large-scale structured and semi-structured data processing.- Proficient across all phases of the Software Development Life Cycle (SDLC), from Requirement Analysis to Support and Documentation.- Extensive experience with Azure cloud technologies, such as Azure Data Lake Storage, Azure Data Factory, and Azure Synapse Analytics.- Solid knowledge of AWS services, including EMR, Redshift, S3, EC2, and auto-scaling configurations.- Skilled in translating business requirements into analytical models, designing algorithms, building models, and developing data mining and reporting solutions for structured and unstructured data.- Proven ability to resolve production issues.- Hands-on experience with data aggregation through Kafka, HDFS, Hive, Scala, and Spark Streams on AWS.- Proficient in working with Big Data tools on AWS, including EC2, EMR, S3, Glue, DynamoDB, and RedShift.- Expertise in developing, deploying, and managing event-driven and scheduled AWS Lambda functions.- Utilization of AWS Data Pipeline for data extraction, transformation, and loading, along with data visualization using Python Matplotlib.- Experience with Cloudera and Hortonworks distributions.- Strong skills in data modeling, including Star-Schema and Snowflake Schema Modeling.- Integration of Spark Streaming with Kafka for real-time data processing.- Designing scalable architectures with Azure components like HDInsight, Data Factory, Data Lake, and ML Studio.- Proficiency in conducting Proof of Concepts (PoCs), gap analysis, data exploration, and data munging using Teradata.- Development of customized Python User-Defined Functions (UDFs) for Hive and Pig Latin.- Expertise in Data Analysis, Profiling, Integration, Migration, Data Governance, Metadata Management, Master Data Management (MDM), and Configuration Management.- Utilization of Spark Streaming APIs for real-time data transformations and persistence to Snowflake.- Strong SQL skills across multiple dialects, including MySQL, PostgreSQL, SQL Server, and Oracle.- Hands-on experience with NoSQL databases like HBase and DynamoDB.