Lead Data Engineer
• Work with stakeholder across the product and tech group in improving the client media data and evaluate right data pipelines and platforms to execute the data flow.• Perform Exploratory Data Analysis of client data using SQL and/or Python/Spark and perform data matching with internal data using Airflow DAG and ingest the results into Data Lake• Built data pipelines, using python, spark, and airflow DAG, which would ingest TBs of data into Data Lake which reduces manual efforts and enables data scientists drive insights. • Work in collaboration with data scientists to transform data and integrate algorithms and models into automated processes.• Build and maintain visualizations in Tableau and Redash.• Define and establish standards and governance in big data solutions and data engineering.