Data Engineer
CurrentWorking on extracting and processing data from various resources, such as web pages, S3 files, databases with a custom data normalization. The dataflow is managed and controlled with the help of Apache Airflow.Responsibilities• Create ETL pipelines• Write scripts for scraping data with Python• Write scripts for data normalization• Write tests• Support legacy code• Write documentation• Code review• Supporting existing DAGs