Data Pipeline DevelopmentCollect and centralize data from over seven companies. Design, create, and manage data pipelines to extract, transform, and load (ETL) data from various data application areas into data warehouseData ArchitectureBuild and develop data mart, data warehouse; maintain relational databases and data warehouse architecture.Data Quality AssuranceImplement data validation and quality checks to ensure the accuracy and consistency of data throughout the ETL process.Dataflow OperatingDevelop and maintain automated workflows using Apache Airflow to schedule and orchestrate data processing tasks, ensuring timely and reliable data delivery.Monitor and troubleshoot the existing data pipelines and build their successors. Design, build, and run the data pipelines using big data and frameworks: Spark, Hadoop, KafkaData PlatformDevelop tool’s feature: condition filter, fulfillment data, create ticket, approval flow to support data governance