Data Engineer
CurrentAt Neoway, I am part of a team responsible for ensuring the integrity and efficiency of the data pipeline, from data ingestion to final client delivery. We work with a modern tech stack, hundreds of data sources, and a microservices architecture, which requires engineers to manage multiple solutions and programming languages. I excel in creating innovative solutions for complex problems, continuously improving services to enhance accessibility for all developers, and focusing on cost reduction, scalability, performance improvement, and resilience.Some highlights of my journey so far include:- Evolving a domain-specific language for data transformation.- Developing a cookie manager for data scraping, which facilitated easier access to complex sources.- Implementing innovative techniques for data scraping, handling JavaScript without the need for browser automation, thus improving efficiency and reducing dependencies on external tools.