Data Engineer- Designed and implemented a comprehensive end-to-end data pipeline and analytics solution over a four-week timeline, showcasing expertise in data ingestion, processing, and storage within a cloud environment.- Ingested raw data from CSV files into Kafka, efficiently streaming it through Spark Structured Streaming for real-time processing.- Utilized the Medallion architecture to stage data effectively, storing structured and cleaned data in Azure Blob Storage for scalable and reliable data management.- Employed Azure Data Factory to orchestrate and load processed data into Azure SQL Database, ensuring seamless integration with downstream analytical workflows.- Conducted thorough data exploration and preprocessing, followed by machine learning model development to derive actionable insights.- Built an intuitive interface to enable natural language queries, enhancing accessibility to data insights.Leveraged Tableau to design interactive dashboards and visualizations, providing clear, data-driven insights.