Senior Software Engineer
Current- Spearheading data ingestion services aimed at empowering users to visualize authentic journeys, measure behavioral impact on metrics, and derive insightful data.- Pioneering the development of a robust ETL pipeline from inception, facilitating the ingestion of data from raw files.- Collaborating closely with the support team to investigate and resolve production issues efficiently.- Driving the implementation of roadmap features designed to enhance ETL pipelines, contributing to continuous improvement initiatives.- Proactively optimizing ETL implementations through the strategic revision of system architecture and the substantial refactoring of codebase sections.- Creating datasets for Generative AI applications, including GPT models. Noteworthy Achievements: -- Successfully mitigated client-provided data file duplication. -- Achieved a notable 10% reduction in ETL pipeline latency. -- Innovatively implemented a fanout service utilizing WebSockets technology. -- Established fault tolerance and recovery mechanisms for enhanced system reliability.Technologies:Python, Flask, Tornado, MySQL, Docker, AWS(EC2, S3,SQS), Bash