disaggregation 是千卡规模的问题
disaggregation 是千卡规模的问题
英文原标题: Disaggregation Is a Thousand-GPU Problem
拆分 prefill 和 decode 需满足三个条件才划算；低于该阈值时，chunked prefill 是更优的默认方案。
Original Article
Three conditions that must hold before splitting prefill from decode pays off, and why chunked prefill is the right default below that threshold. The post Disaggregation Is a Thousand-GPU Problem appeared first on Towards Data Science.