Airflow-based batch pipeline operation in EKS environmentBuilt and operated TB-scale DW and data mart based on dbt and BigQueryBuilt and operated 10+ Kafka CDC pipelines (MSK, Strimzi connect on EKS)Built and operated event-driven Kafka consumer applicationsDeveloped Flink applications with AWS Managed FlinkBuilt Prometheus/Grafana based monitoring environment (helm, EKS)Built and operated Clickhouse real-time OLAP data warehouse

Designed and implemented a churn prediction pipeline for 3,000+ games, automating model training, deployment, and monitoring, with real-time metrics visualization.Processed 50-200 GB of semi-structured data daily using Airflow and Google Cloud Dataproc, storing results in Parquet format for efficient querying.Introduced and deployed Airflow(AWS MWAA) to manage 20+ DAGs, reducing manual maintenance by 30% with fully automated CI/CD pipelines.Designed an API log aggregation pipeline handling 300 million daily records, achieving end-to-end processing within 1 minute, and storing partitioned data in S3.Built an ETL pipeline to storing 1+ TB/month of ElasticSearch data in S3 as Parquet, enabling Athena query access.