About

I’m Raymond — Yu-Chuan Hung — a data engineer based in Taipei.

Work

I build lakehouse platforms on Spark and Iceberg, running on Kubernetes:

  • Batch & streaming ingestion — 900M+ records a day into 500+ production tables, including CDC streaming (Kafka + Spark Structured Streaming) with sub-3-minute freshness.
  • Catalog migration — moved 800+ Iceberg tables to Apache Polaris (Iceberg REST), removing commit failures from non-atomic S3 renames.
  • Developer experience — YAML-driven pipeline generation that cut source onboarding from 6–8 hours to 3 minutes.

Full CV: PDF

Open source

I contribute to Apache DataFusion and Apache Arrow (arrow-rs) in Rust — 15+ merged PRs so far. A few highlights:

  • Join execution — sort-merge join filter evaluation on NULL-padded rows (#26027), HashJoinExec probe metrics (#25272)
  • Performance — type-specialized scatter, 1.5–2.5× faster (#20498); cached dictionary slots in aggregation (#25652)
  • Spark compatibility — size (#19592) and json_tuple (#20412)
  • Parquet — Bloom filter properties builder in arrow-rs (#9877, write-up)

All merged PRs: DataFusion · arrow-rs

Contact

Find me on GitHub, LinkedIn, or by email.

Write to remember, publish to be remembered.