- Who it helps
- Inference engineers studying models too large for a single accelerator
- Availability
- November 8 paper is public. Its reported DeepSeek-V4-Flash experiment uses four RTX 4090D GPUs; runnable artifact availability was not checked.
A hardware-specific result
A mixture-of-experts model selects parts of its network for each token. The authors report reducing waiting time by keeping repeatedly useful experts available on the GPUs.
Check the deployment boundary
The experiment is evidence about the tested multi-GPU configuration. It does not establish a practical single-GPU setup or the same improvement on other workloads.
What you can do
- Read the primary description and its stated limits before planning an evaluation. Open reference ↗
Where the evidence stops
Abstract checked November 12. No runtime installed or latency result reproduced.
Sources read. Public source text checked November 12, 2026 UTC. No third-party project or full workflow executed.
Sources & further reading
Paper · paper
Ostinato paper ↗
November 8 abstract read; four-GPU experiment distinguished from single-device inference.
Junning Zhu and coauthors · page checked 2026-11-12 · Published 2026-11-08
Coverage note: Abstract checked November 12. No runtime installed or latency result reproduced.
The record over time (1)
Dated source checks and published explanations. An observation date can differ from the original release date.
- · Added to Current
Ostinato keeps selected model experts on GPUs while coordinating the remaining work with CPU memory. It targets delays across both prompt processing and token generation.