Data Engineering Manager - Data Platform
CurrentLeading/Managing Data Engineering and Software Engineering Teams that provide data to several vertical business units across Red Ventures. Responsible for developing, enhancing, maintaining the cloud data platform to ingest the data from 300+ sources, 1 billion events every day, and provide data to vertical & horizontal business units for activation, reporting and decision making use cases. To enhance our data processing capabilities and accelerate time-to-insights, We spearheaded the… Show more Leading/Managing Data Engineering and Software Engineering Teams that provide data to several vertical business units across Red Ventures. Responsible for developing, enhancing, maintaining the cloud data platform to ingest the data from 300+ sources, 1 billion events every day, and provide data to vertical & horizontal business units for activation, reporting and decision making use cases. To enhance our data processing capabilities and accelerate time-to-insights, We spearheaded the adoption of Data Mesh & Lakehouse architecture using AWS & Databricks features. This initiative resulted in a significant reduction in data latency from several hours to just 7 minutes, while also achieving a 40% reduction in costs through optimization efforts.We leverage a mix of cutting-edge commercial and open-source technologies, along with custom-built in-house tools, to meet the evolving demands of our data infrastructure.Our platform serves as the backbone for all of Red Ventures' global businesses, including well-known brands such as CNET, Bankrate, Lonely Planet, The Points Guy, NextAdvisor, uSwitch, and many others. Show less
