Prefill delay
GPUs
1
2
4
Poisson
Even
Pause
40 requests · Poisson arrivals
0.00 s
01 · Find the CEO of Lucid Motors
Arriving
01 · Classify
…
→
02 · Policy
λ =
7.5
→
03 · Utility
Nano
…
Mini
…
Sol
…
→
04 · Route
…
Illustrative timings and numerics