Highly skilled Data Engineer with 8 years of experience in developing and managing scalable data pipelines, ensuring data quality, and enabling data-driven decision-making. From data intake using ETL tools like Apache Kafka and AWS Glue to data warehousing via technologies like Snowflake and Amazon Redshift, my expertise covers the whole data ecosystem. Have experience in creating and improving data pipelines for processing massive amounts of data using tools like Apache Spark, Hadoop, and Flink, also created and administered data lakes and data warehouses on cloud platforms like AWS and Azure, assuring scalability and high availability. Knowledge in real time analytics with Apache Spark (RDD, Data Frames and Streaming API). Developed and maintained scalable data processing pipelines on AWS, Azure using Python and Pandas, ensuring efficient ETL (Extract, Transform, Load) operations for large volumes of structured and unstructured data. Good experience in Snowflake data modeling, ELT using Snowflake SQL, Snowflake Task Orchestration implementing complex stored Procedures, and standard DWH and ETL concepts. Possess hands-on experience in connecting Azure Databricks with Azure Synapse Analytics to conduct data analysis utilizing Apache Spark and then visualizing the results using Power BI. Adept at automating workflows and ensuring operational efficiency with tools like GitLab CI/CD, Selenium, and AWS Lambda.