Software Engineer
Current
Taipei, Taipei City, Taiwan
- Led the development of an LLM-powered service, implementing a robust prompt engineering pipeline with Haystack and FastAPI. Integrated OpenTelemetry, Google Cloud Trace for comprehensive monitoring of usage, costs, and.
- Designed and implemented an LLM batch prediction framework using Airflow’s dynamic task mapping and deferrable operators, optimizing scalability for large-scale user segmentation with Trino. Successfully integrated.
- Led end-to-end development of a scalable error handling microservice supporting 7 connection types (S3 bucket, S3 file, GCS, SFTP, BigQuery, Google Sheets). Delivered Airbyte API integration and successfully deployed.
- Resolved a longstanding pod creation issue in Airbyte data integration that previously required manual restarts. Setup OpenTelemetry metrics collection to proactively alert on failed connection creation, enhancing.
- Designed and implemented integration across 6 feature components using the Feature Flag Decorator pattern, ensuring code maintainability and enabling ease of unit testing while accelerating feature delivery.
- Designed offline event segmentation and data preprocessing workflows leveraging Celery, Trino, and GraphQL. Accelerated feature delivery by 60% and achieved 100% key results completion, including custom segmentation.