Data Architect
Segway is an international hi-tech company with a mission to ‘Simplify Moving’. Skills: Java, Scala, SQL, Spark, Kafka, Hive, Hadoop, Apache Doris, FineBI, ELK, AWS, Internet of Vehicles(IoV).Highlights:1. Built a data warehouse with 1,100+ tables in 6 tiers: STG/ODS/DWD/DIM/DWS/ADS, occupying 600+ TB of storage, including daily data pipelines consisting of 1,300+ tasks (Hive tasks: 56%, ETL tasks: 29%, Shell tasks: 12%, Spark tasks: 3%), supporting 100+ BI dashboards for… Show more Segway is an international hi-tech company with a mission to ‘Simplify Moving’. Skills: Java, Scala, SQL, Spark, Kafka, Hive, Hadoop, Apache Doris, FineBI, ELK, AWS, Internet of Vehicles(IoV).Highlights:1. Built a data warehouse with 1,100+ tables in 6 tiers: STG/ODS/DWD/DIM/DWS/ADS, occupying 600+ TB of storage, including daily data pipelines consisting of 1,300+ tasks (Hive tasks: 56%, ETL tasks: 29%, Shell tasks: 12%, Spark tasks: 3%), supporting 100+ BI dashboards for data-driven decision-making.2. Achieved significant cost savings, €100,000 per year, by optimizing and tuning Hive/Hadoop/HBase/Kafka/ELK to free up 300+ CPU cores, 800+ GB memory, and 400+ TB disk.3. Improved customer experience by introducing an intelligent push notification service based on a 200-tag customer profiling system, and engineering a model to make accurate predictions like remaining mileage and charge time predictions.4. Developed incorrect IoV data monitoring, big tables fully synchronization optimization, and customer de-duplication.5. Recruited 6 data engineers and 1 data analyst to enlarge the big data team. Show less