Data Engineer
Key Achievements:• Implemented a comprehensive Data Governance framework using DataHub, ensuring data quality, integrity, and accessibility.• Developed and maintained streaming data pipelines leveraging Apache Spark and Kafka, optimizing real-time data processing.• Integrated data from multiple sources, including databases and APIs, to ensure a unified data ecosystem.• Designed and implemented a Data Lake infrastructure utilizing S3, AWS Redshift, and GreenPlum, enhancing data storage and accessibility.• Utilized tools such as Tableau, Streamlit, PostHog, SciPy, Statsmodels, and PyMC for data visualization, statistical analysis, and machine learning.Results: Developed and deployed data analytics platforms for multiple clients, significantly improving decision-making processes through data-driven insights. Conducted in-depth research on user behavior, leading to a 5% increase in conversion rates by optimizing product value. Additionally, performed machine learning-driven analysis of marketing campaigns to enhance strategic targeting and effectiveness.
