dual-radeon-vllm · write-ups

Two consumer Radeons,
measured

Long-form write-ups from a repository of measurements taken on 2× RX 7900 XT (gfx1100, RDNA3) under ROCm and vLLM. Each article draws its figures from the same committed data the repository verifies, and says on the page which of them can be recomputed and which cannot.

2× RX 7900 XT · gfx1100 · TP=2 VFIO guest · no P2P · PCIe 3.0 cross-die repository

1What this machine does today

One line per model per machine, each of them that model's fastest measured configuration rather than a common one. Four of the six Radeon lines come from a single campaign on one stack and are directly comparable; the other two are drawn from a later stack, because on those two models the campaign is no longer the best this machine can do. The A100 carries five of the same models, its own whole campaign on one afternoon. Every line says which it is.

Figure 1 · decode rate at batch 1, best measured configuration per model recomputable from repo

compare

Figure 1. Decode rate at batch 1 against context length, log axis. The default view is every model on 2× RX 7900 XT at tensor parallelism 2; the switch above the chart turns the question round, to one model on every machine, and colour follows the question: the model in one view, the card in the other.

Eleven machines: The pair; one of its cards; an A100 80G, an L4 24G and a Tesla T4 16G from Colab; and six rented on 2026-09-03 — an H100, an H200, a B300, an RTX PRO 6000, and two H100s and two PRO 6000s at TP=2. The checkpoint is what they share — vLLM 0.28.0 on every CUDA card against 0.23.1 on the pair. The split and the stack are not shared, and every line's tooltip carries its own.
The T4: It is here at all only because of vllm#39018: without it the engine dies at kernel load asking 98 304 bytes of shared memory against Turing's 65 536. That patch changes prefill's tile size and nothing else, so this figure reads its line on the same terms as every other. Figure 2 does not, and says so.
Lines that stop: Four do, and each stop is arithmetic rather than an abandoned run: a card that cannot hold the KV for the next rung cannot measure it, and the tooltip says how many tokens its KV held. The shortest is two rungs — gemma-4-31B does not start on the L4 at all with CUDA graphs on, and --enforce-eager buys it 2.51 GiB and a 2 020-token pool.
The switches: Each is named for what the engine resolved rather than for the button. Speculation shows what happens to that line — same day, same kernel — and one arm says DFlash because Muse-Glimmer's drafter is block-diffusion and not MTP. It does not go one way: on these cards both arms start ahead and one stays ahead to 32 K, while on the A100 all four lose at every depth. Qwen3.8's second switch, ROCM_ATTN, adds the backend the runtime picks for itself; the flag this repository pins instead buys decode and sells prefill by more (Figure 2).
Deliberately absent: Two series this repository measured. The L4's Qwen3.8-27B is a different checkpoint from the one drawn — symmetric compressed-tensors against AWQ, 1.27–3.24× apart on decode on gfx1100 — and a legend keyed on the model name would put both behind one entry. Qwen3-8B on one 7900 XT under vLLM 0.27 decodes identically to the line already there, to 0.21 % at every rung, and swapping it in would turn Figure 2's second-card comparison into a stack comparison.

Figure 2 · prefill throughput, every model with a ladder, eleven machines recomputable from repo

compare

Figure 2. The other half of the same question: Figure 1 is how fast each machine generates, this is how fast it reads the prompt. Six models, fifty-two lines, eleven machines — every chart-grade prefill ladder this repository has between 500 and 32 000 tokens, recorded beside decode since July and never drawn until now. Colour and stroke are Figure 1's.

Why two terms: Prefill is not one number. Split the time into a fixed cost, a linear term and attention's quadratic one — T(S) = a + b·S + c·S² — and b is compute while c is how badly attention scales. Hover a point for that line's fitted pair.
Two lines re-measured: 2026-09-02. The pair's gemma-4-31B and Qwen3.8-27B lines were first measured on 2026-08-29, when one card had trained at PCIe 3.0 x8 — visible in the host's boot journal, never in the guest's sysfs. A reboot restored it and both arms ran again, same container, same serve arguments: b 868.7 → 722.6 on the 31B and 846.2 → 758.5 on Qwen3.8. Those are the lines drawn.
Card against card: On the dense 12B the A100 is 3.3× on b and 6.7× on c against one 7900 XT, so the gap in attention is twice the gap in compute. The L4 is slower on b (0.90×) and 3.0× better on c, which is why its curve starts below the Radeon's and ends above it.
The second card: On all three models with both topologies it buys 1.2–1.5× on b and 1.9–2.2× on c. Attention parallelises better than the GEMMs because it needs no communication.
The flag is a trade: Pinning Qwen3.8 to TRITON_ATTN, which this repository called free until 2026-08-30, is worth 1.15× of decode at 32 K and costs 1.45× of prefill there.
One line is different: The T4 runs gemma-4-12B only with vllm#39018, which halves TILE_PREFILL on the head_size 512 layers — the quadratic term this figure decomposes. Its c is this card with a different kernel, so it is off until asked for and left out of the ratios above. Its b is not determined by its own ladder either: the 32 K rung came from a second VM, the two agree to 4.61 %, and swapping which supplies it moves b by 29.9 %. This curve is quadratic-dominated where no other here is — 224 s against 97 s at 32 K — so the linear term absorbs the uncertainty, as a does everywhere.
Dropped rungs: Only rungs whose two rounds agree are drawn, and the one each line drops is not one story: six drop their shallowest — on the CUDA cards a cold first request, not discarded until 2026-08-30 — and three Radeon lines drop the 4 000 rung, round 1 slower each time, which this repository does not explain. The MoE on one 7900 XT stops at 12 K for a reason it does know: 16.96 GiB of weights leave 0.93 GiB of KV, 13 149 tokens.

Every ladder on this machine stopped at 32 000 tokens until 2026-09-03, and that read as the machine's limit. It was the campaign's: each arm set max_model_len to 33 000 while its KV pool held several times that. The two figures below are the same two questions asked to 128 000 — the pair as the subject, on one ladder cut for it that day, and the machines rented the same day one click away as the background.

Figure 3 · decode rate to 128 000 tokens, the pair and the rented machines recomputable from repo

compare

Figure 3. Decode rate at batch 1 against context length, 500 to 128 000, log axis. The default view is 2× RX 7900 XT at tensor parallelism 2 on the 2026-09-03 ladder — the first rows on this machine past 32 000 — and the switch above the chart puts one model on every machine rented that day instead. The controls are Figure 1's.

The whole ladder: Sixteen rungs, 500 upward, rather than only the five past 32 000: a curve's shape at depth is only readable against where it started. Where a rung is on both this ladder and Figure 1's, the two are different sittings on different prompt cuts, and the tooltip carries each point's own date.
Deliberately absent: Qwen3-8B, on every machine. Its own config caps context at 40 960, so its ladder is Figure 1's eleven rungs everywhere it was run — a ceiling the model carries, not the campaign.

Figure 4 · prefill throughput to 128 000 tokens, the pair and the rented machines recomputable from repo

compare

Figure 4. Prefill throughput against prompt length to 128 000, the same lines as Figure 3. T(S) = a + b·S + c·S² is fitted over the whole ladder here, and the note under the chart says on each of the pair's lines where c·S² overtakes b·S and what share of the deepest rung it is.

Two fits for one arm: Figure 2 fits each line on the rungs it draws, to 32 000; this figure fits the sixteen. For the same arm those are two different pairs of coefficients, and each tooltip says which it is carrying.
Dropped rungs: Only rungs whose two rounds agree are drawn, as in Figure 2, and each line's tooltip names the rung it dropped and how far apart the rounds were.

2Articles

Eleven write-ups, newest first by the date on the card — the last day evidence for that article was taken here, reported upstream or re-read. Under each title is the line that article used to lead with; under that, what it establishes, in the words the synthesis piece uses to classify it, so the two cannot describe a finding differently.

3What each of these is dated to

The date on an article is the date its evidence was taken, not the date it was written. Three of the eleven are dated to the last week of July and eight to the last week of August. The machine itself ran on 25, 26 and 28 July and on 1 August, then not again for three weeks — the second cluster is largely re-measurement of the first under conditions the first got wrong. A bar spanning two dates is an article whose evidence comes from two separate sittings, not one that ran throughout.

Figure 5 · the date each article carries the dates the articles carry

Figure 5. Positioned by date, not by publication order. Two of the eleven are not measurements of this machine: one records the date a defect was reported upstream, and the synthesis re-read sixteen upstream threads rather than running anything. Both are marked as such rather than given a measurement date they do not have.

4How to read these

Each article is a dated record, not standing advice. Two of the largest findings in this repository have already moved since they were measured: an Ubuntu kernel regression was fixed in 7.0.0-30, and a loader speed-up of 3.9× to 5.6× was withdrawn when page cache was properly controlled. Every article therefore closes with a section listing what has changed since, which is appended to rather than rewritten.

Figures carry a provenance marker. recomputable from repo means verify_doc_figures.py checks that value against its source file and fails non-zero if the two disagree — the same check that guards the repository's own documents. raw output not committed means the number is transcribed from a run whose raw output is not in the repository, and cannot be recomputed.

The absolute rates are a floor rather than a benchmark of the hardware: this is a VFIO guest with no peer-to-peer between the cards and a cross-die PCIe 3.0 path, chosen deliberately as the least favourable topology. The shapes transfer; the levels are the worst case.

Source, raw data and the runners that produced the data: github.com/cadamcat/dual-radeon-vllm. Upstream reports and patches are linked from each article.