Data Engineer
CurrentLed End-to-End Data Migration & Transformation:- Successfully migrated and transformed 800+ objects totaling 4TB from SQL Server to Snowflake, optimizing data processing capabilities for Sams Club.Success Metrics: Achieved a 30% increase in processing speed by implementing Spark code using Scala and Spark-SQL/Streaming. Enabled seamless data migration from various sources like AWS Redshift, Oracle, MongoDB, and SQL Server, enhancing data accessibility and boosting brand's technological prowess.Architected and Implemented Large-Scale Data Intelligence Solutions:-Designed and developed a robust data warehousing solution using Snowflake, integrating AWS, Kafka, Spark, Scala, and various big data technologies, to align with Sams Club's analytical needs.Success Metrics: Reduced data access time by 25% through intelligent query streamlining and implementing Redis to support Snowflake. Enhanced data sharing between two Snowflake accounts, strengthening internal collaboration and establishing a reputation for innovative data management solutions within the industry.Developed Real-Time Stream Processing Applications & APIs:-Created a real-time stream processing application using Spark, Kafka, Scala, and Hive for Streaming ETL, and developed internal APIs with Node.js, MongoDB, Python, Flask, and Django to enhance the user interface and operational efficiency.Success Metrics: Achieved a 20% increase in real-time data processing capabilities and contributed to a 15% growth in user engagement on the internal platforms. This development positioned Sams Club as a frontrunner in technological innovation, enhancing brand recognition and trust within the market.