Skip to content
Development environment

You're on the development site

This is a development environment, not for production use. If you're a potential customer, please continue on our main site.

Go to the main siteonenexus.cloud

Model Release

GLM-5.3-Flash in MXFP4: A Lighter, Faster Checkpoint for AMD MI355X

We published an MXFP4 GLM-5.3-Flash checkpoint tuned for AMD Instinct MI355X. Against the official FP8 checkpoint on the same four GPUs, it is 40% smaller, with 12.4% more output throughput and an 18.2% faster P90 per-user rate (geometric means, c1–c48).

+18.2%
Per-user speed vs FP8
+12.4%
Throughput vs FP8
−40%
Checkpoint size

All articles GLM-5.3 AMD Inference Quantization

4-bit routed experts, a serving path tuned kernel by kernel, and every number published with its protocol. This is what it took to run GLM-5.3-Flash well on AMD Instinct MI355X.

GLM-5.3-Flash is a mixture-of-experts model with 320B total and 18B active parameters, a 1,048,576-token context limit, and text-and-image input. We quantized its routed experts to MXFP4, tuned the SGLang serving path for gfx950, and published the result under the MIT license as OneNexus/GLM-5.3-Flash-MXFP4 on Hugging Face.

The headline is one picture. On the same four MI355X GPUs, with the same SGLang source, placement, KV policy, speculative-decoding shape and AgentX workload, the MXFP4 checkpoint sits outside the official FP8 checkpoint at every concurrency level from c1 to c48: more aggregate output and a faster per-user rate at the same time.

Figure 1

MXFP4 moves the whole serving frontier outward

Serving frontier: official FP8 versus OneNexus MXFP4 on four MI355X GPUs Aggregate output tokens per second against P90 output tokens per second per user, concurrency 1 to 48. The MXFP4 curve lies above and to the right of the FP8 curve at every level, from 262 versus 248 tok/s at c1 to 2,356 versus 1,987 tok/s at c48. Official FP8 checkpoint · c1 — Output tok/s: 247.67 · P90 tok/s/user: 339.50 Official FP8 checkpoint · c2 — Output tok/s: 419.27 · P90 tok/s/user: 291.18 Official FP8 checkpoint · c4 — Output tok/s: 714.35 · P90 tok/s/user: 251.81 Official FP8 checkpoint · c8 — Output tok/s: 1,099.33 · P90 tok/s/user: 191.91 Official FP8 checkpoint · c16 — Output tok/s: 1,547.01 · P90 tok/s/user: 145.77 Official FP8 checkpoint · c32 — Output tok/s: 1,859.89 · P90 tok/s/user: 86.37 Official FP8 checkpoint · c48 — Output tok/s: 1,986.75 · P90 tok/s/user: 63.80 OneNexus MXFP4 · c1 — Output tok/s: 262.19 · P90 tok/s/user: 380.35 OneNexus MXFP4 · c2 — Output tok/s: 441.34 · P90 tok/s/user: 337.36 OneNexus MXFP4 · c4 — Output tok/s: 768.45 · P90 tok/s/user: 294.51 OneNexus MXFP4 · c8 — Output tok/s: 1,269.90 · P90 tok/s/user: 238.61 OneNexus MXFP4 · c16 — Output tok/s: 1,831.06 · P90 tok/s/user: 176.74 OneNexus MXFP4 · c32 — Output tok/s: 2,175.11 · P90 tok/s/user: 106.55 OneNexus MXFP4 · c48 — Output tok/s: 2,355.83 · P90 tok/s/user: 73.01 Serving frontier: official FP8 versus OneNexus MXFP4 on four MI355X GPUs Aggregate output tokens per second against P90 output tokens per second per user, concurrency 1 to 48. The MXFP4 curve lies above and to the right of the FP8 curve at every level, from 262 versus 248 tok/s at c1 to 2,356 versus 1,987 tok/s at c48. Official FP8 checkpoint · c1 — Output tok/s: 247.67 · P90 tok/s/user: 339.50 Official FP8 checkpoint · c2 — Output tok/s: 419.27 · P90 tok/s/user: 291.18 Official FP8 checkpoint · c4 — Output tok/s: 714.35 · P90 tok/s/user: 251.81 Official FP8 checkpoint · c8 — Output tok/s: 1,099.33 · P90 tok/s/user: 191.91 Official FP8 checkpoint · c16 — Output tok/s: 1,547.01 · P90 tok/s/user: 145.77 Official FP8 checkpoint · c32 — Output tok/s: 1,859.89 · P90 tok/s/user: 86.37 Official FP8 checkpoint · c48 — Output tok/s: 1,986.75 · P90 tok/s/user: 63.80 OneNexus MXFP4 · c1 — Output tok/s: 262.19 · P90 tok/s/user: 380.35 OneNexus MXFP4 · c2 — Output tok/s: 441.34 · P90 tok/s/user: 337.36 OneNexus MXFP4 · c4 — Output tok/s: 768.45 · P90 tok/s/user: 294.51 OneNexus MXFP4 · c8 — Output tok/s: 1,269.90 · P90 tok/s/user: 238.61 OneNexus MXFP4 · c16 — Output tok/s: 1,831.06 · P90 tok/s/user: 176.74 OneNexus MXFP4 · c32 — Output tok/s: 2,175.11 · P90 tok/s/user: 106.55 OneNexus MXFP4 · c48 — Output tok/s: 2,355.83 · P90 tok/s/user: 73.01
Each point is one 900-second concurrency level (c1–c48) of the 393-entry AgentX corpus on four MI355X GPUs. Up and to the right is better; the shaded band is the measured gap between the two checkpoints. Point tooltips carry the values, and section 02 prints them in full. Source: OneNexus/GLM-5.3-Flash-MXFP4 model card · SGLang 5fd4387a · TP4/EP4 · FP8 KV cache · EAGLE 5/1/6

01The model at a glance

Base model
zai-org/GLM-5.3-Flash 320B total / 18B active parameters
Weight format
MXFP4 routed experts In most layers; other tensors keep their declared precision
Checkpoint
182.12 GiB Indexed tensor payload: 195,550,624,632 bytes
Context limit
1,048,576 tokens The AgentX runs below used a 262,144-token cap
Tested serving stack
4× MI355X · SGLang on ROCm · TP4/EP4 · FP8 KV cache In-checkpoint MTP head with EAGLE 5/1/6 speculative decoding
Input / output
Text and image → text Image request and cached follow-up tested; video not evaluated
License
MIT Source attribution in NOTICE

Runtime support lives upstream in SGLang PR #39273, which carries the implementation and its correctness tests.

02What four bits buy over FP8

Same GPUs, same software, same workload, two complete checkpoints. The MXFP4 one is smaller on disk, lighter in memory, and faster at every concurrency level from c1 to c48.

Both curves in Figure 1 were measured with SGLang source 5fd4387a on the same image, GPU set, tokenizer, TP4/EP4 placement, FP8 KV policy, EAGLE 5/1/6 shape, graph limit, cache reset, 393-entry AgentX corpus and 900-second rungs. Over c1–c48, MXFP4 delivers +12.4% aggregate output rate and +18.2% P90 per-user output rate as geometric means. Neither arm had a single request error: 9,734 requests on FP8 and 11,810 on MXFP4 across all eight measured rungs.

Figure 2

The gain is largest under load

Uplift of MXFP4 over FP8 by concurrency
Metric c1c2c4c8c16c32c48
Output tok/s +5.9%+5.3%+7.6%+15.5%+18.4%+16.9%+18.6%
P90 tok/s/user +12.0%+15.9%+17.0%+24.3%+21.2%+23.4%+14.4%
Relative change of OneNexus MXFP4 over the official FP8 checkpoint at each concurrency level. Per-user speed gains the most in the middle of the curve (c8–c32); aggregate output gains the most at c16 and above. Source: matched FP8 / MXFP4 AgentX replay, model card

The smaller checkpoint also frees memory for context. With the same memory-allocation policy, weights drop from 77.69 to 46.99 GB per GPU, and device KV capacity rises from 13.97M to 17.32M tokens.

Figure 3

Smaller files, lighter GPUs, more room for KV cache

  • Checkpoint payload −40.4%
    Official FP8: 305.78 GiB
    OneNexus MXFP4: 182.12 GiB
  • Weights per GPU −39.5%
    Official FP8: 77.69 GB
    OneNexus MXFP4: 46.99 GB
  • Device KV capacity +24.0%
    Official FP8: 13.97M tokens
    OneNexus MXFP4: 17.32M tokens

Each row is scaled to its own larger value.

The 40.44% reduction is in checkpoint files (indexed tensor payload), not a peak-memory comparison. Weights and KV capacity were measured under the same memory-allocation policy. Source: model card, FP8 / MXFP4 AgentX control summaries
ConcurrencyFP8 output tok/sMXFP4 output tok/sUpliftFP8 P90 tok/s/userMXFP4 P90 tok/s/userUplift
1247.67262.19+5.9%339.50380.35+12.0%
2419.27441.34+5.3%291.18337.36+15.9%
4714.35768.45+7.6%251.81294.51+17.0%
81,099.331,269.90+15.5%191.91238.61+24.3%
161,547.011,831.06+18.4%145.77176.74+21.2%
321,859.892,175.11+16.9%86.37106.55+23.4%
481,986.752,355.83+18.6%63.8073.01+14.4%

What this comparison does and does not isolate

The two complete checkpoints and their declared MTP precision differ, so this compares checkpoints — it does not isolate quantization-kernel speed. It is also a separate matched replay from the AMD comparison in section 04, which is why its OneNexus rates differ slightly. Dominated c64 observations appear in the model card’s figure for context but are excluded from the reported uplift.

03Reasoning quality next to AMD’s MXFP4

A 4-bit checkpoint is only worth serving if it still reasons. We put ours next to AMD’s reference MXFP4 checkpoint on the same fixed questions.

Both checkpoints — ours and amd/GLM-5.3-Flash-Quark-MXFP4 — answered the same fixed questions with one greedy attempt per question. A capped or unfinished response counts as incorrect. On these samples ours scores +1.0 point on MMLU and +3.5 points on GPQA Diamond, with −0.2 point on GSM8K.

Figure 4

Accuracy on matched text-reasoning samples

  • GSM8K · 500 questions −0.2 pp
    AMD reference MXFP4: 98.2%
    OneNexus MXFP4: 98.0%
  • MMLU · 500 questions +1.0 pp
    AMD reference MXFP4: 84.6%
    OneNexus MXFP4: 85.6%
  • GPQA Diamond · 198 questions +3.5 pp
    AMD reference MXFP4: 66.7%
    OneNexus MXFP4: 70.2%
One greedy attempt per question; capped or unfinished answers count as wrong. Bars start at zero, so small differences look small — which is the honest reading. Source: model card, per-question aggregate data (checkpoint-quality-comparison.json)
SampleOneNexus correctAMD correctObserved difference
GSM8K (500 questions)490 (98.0%)491 (98.2%)−0.2 pp
MMLU (500 questions)428 (85.6%)423 (84.6%)+1.0 pp
GPQA Diamond (198 questions)139 (70.2%)132 (66.7%)+3.5 pp

Read these as observations, not a win

MMLU and GPQA are sensitive to the answer-length limit: MMLU had 43/500 truncated outputs for each checkpoint, and GPQA had 55/198 for OneNexus versus 61/198 for AMD. The paired exact-test p-values are 0.458 for MMLU and 0.265 for GPQA — neither reaches p < 0.05, so the samples prove neither superiority nor equivalence. These are fixed-subset text-reasoning evaluations, not leaderboard scores, and they make no claim about vision, safety or general agentic quality.

04Serving speed: parity with AMD

On throughput against AMD’s own MXFP4 checkpoint, the honest word is parity.

We replayed both checkpoints on the same four MI355X VFs with the same SGLang build, tokenizer and prompt rendering, TP4/EP4 placement, FP8 KV cache, EAGLE 5/1/6 settings, cache policy, 393-entry AgentX corpus and 900-second measurement per level; each checkpoint used its own declared target and MTP weight precision. Across c1–c48, OneNexus is 0.1% lower in geometric-mean aggregate output and 1.0% higher in geometric-mean P90 per-user rate. Neither run had a request error (9,223 OneNexus and 9,264 AMD requests).

Figure 5

Two curves, nearly one line

Serving frontier: AMD reference MXFP4 versus OneNexus MXFP4 Aggregate output tokens per second against P90 output tokens per second per user, concurrency 1 to 48. The two curves overlap at every level; at c48 OneNexus serves 2,331 and AMD 2,340 tok/s. AMD reference MXFP4 · c1 — Output tok/s: 258.89 · P90 tok/s/user: 364.96 AMD reference MXFP4 · c2 — Output tok/s: 444.09 · P90 tok/s/user: 330.21 AMD reference MXFP4 · c4 — Output tok/s: 769.09 · P90 tok/s/user: 297.03 AMD reference MXFP4 · c8 — Output tok/s: 1,278.38 · P90 tok/s/user: 237.75 AMD reference MXFP4 · c16 — Output tok/s: 1,857.38 · P90 tok/s/user: 179.39 AMD reference MXFP4 · c32 — Output tok/s: 2,290.16 · P90 tok/s/user: 110.00 AMD reference MXFP4 · c48 — Output tok/s: 2,340.38 · P90 tok/s/user: 73.89 OneNexus MXFP4 · c1 — Output tok/s: 261.76 · P90 tok/s/user: 379.08 OneNexus MXFP4 · c2 — Output tok/s: 444.88 · P90 tok/s/user: 333.60 OneNexus MXFP4 · c4 — Output tok/s: 764.52 · P90 tok/s/user: 295.25 OneNexus MXFP4 · c8 — Output tok/s: 1,283.08 · P90 tok/s/user: 241.23 OneNexus MXFP4 · c16 — Output tok/s: 1,845.35 · P90 tok/s/user: 178.88 OneNexus MXFP4 · c32 — Output tok/s: 2,279.72 · P90 tok/s/user: 110.35 OneNexus MXFP4 · c48 — Output tok/s: 2,331.22 · P90 tok/s/user: 74.85 Serving frontier: AMD reference MXFP4 versus OneNexus MXFP4 Aggregate output tokens per second against P90 output tokens per second per user, concurrency 1 to 48. The two curves overlap at every level; at c48 OneNexus serves 2,331 and AMD 2,340 tok/s. AMD reference MXFP4 · c1 — Output tok/s: 258.89 · P90 tok/s/user: 364.96 AMD reference MXFP4 · c2 — Output tok/s: 444.09 · P90 tok/s/user: 330.21 AMD reference MXFP4 · c4 — Output tok/s: 769.09 · P90 tok/s/user: 297.03 AMD reference MXFP4 · c8 — Output tok/s: 1,278.38 · P90 tok/s/user: 237.75 AMD reference MXFP4 · c16 — Output tok/s: 1,857.38 · P90 tok/s/user: 179.39 AMD reference MXFP4 · c32 — Output tok/s: 2,290.16 · P90 tok/s/user: 110.00 AMD reference MXFP4 · c48 — Output tok/s: 2,340.38 · P90 tok/s/user: 73.89 OneNexus MXFP4 · c1 — Output tok/s: 261.76 · P90 tok/s/user: 379.08 OneNexus MXFP4 · c2 — Output tok/s: 444.88 · P90 tok/s/user: 333.60 OneNexus MXFP4 · c4 — Output tok/s: 764.52 · P90 tok/s/user: 295.25 OneNexus MXFP4 · c8 — Output tok/s: 1,283.08 · P90 tok/s/user: 241.23 OneNexus MXFP4 · c16 — Output tok/s: 1,845.35 · P90 tok/s/user: 178.88 OneNexus MXFP4 · c32 — Output tok/s: 2,279.72 · P90 tok/s/user: 110.35 OneNexus MXFP4 · c48 — Output tok/s: 2,331.22 · P90 tok/s/user: 74.85
The same frontier view as Figure 1, for the matched AMD replay. The curves overlap at every level — a measured performance-parity result, not a throughput win. Source: model card, AgentX summaries for both checkpoints
ConcurrencyOneNexus output tok/sAMD output tok/sOneNexus P90 tok/s/userAMD P90 tok/s/user
1261.76258.89379.08364.96
2444.88444.09333.60330.21
4764.52769.09295.25297.03
81,283.081,278.38241.23237.75
161,845.351,857.38178.88179.39
322,279.722,290.16110.35110.00
482,331.222,340.3874.8573.89

P90 tok/s/user is the 90th percentile of each eligible request’s reciprocal mean inter-token latency: a per-user speed, not aggregate throughput.

05How the serving path works

Two mechanisms carry most of the work: sparse attention over a compact index cache, and speculative decoding with a draft head that ships inside the checkpoint.

Sparse attention and K-pool selection

GLM-5.3-Flash uses 256-wide sparse-attention projections with no RoPE tail. A separate K-pool index cache, distinct from the 512-wide MLA attention cache, decides which history each token attends to. The native gfx950 path reads and writes the same preshuffled index layout, and it keeps selection bounded at long context without dropping valid positions.

Figure 6

Compact index rows, bounded selection

One K-pool index row: 132 bytes

128 FP8 index-key values · 128 B FP32 scale · 4 B

Bounded selection at long context

  • History, pooled into four-token groups
  • Up to 512 selected groups × 4 = 2,048 tokens
  • Up to 3 recent, unpooled tokens

Mapped width: 2,048 + 3 = 2,051

Logical data flow: fused kernels combine these stages, and unused output positions stay masked. The history strip is schematic; the counts in the labels are exact. Source: model card, dsa-kpool-mechanism.json

Draft, verify, commit

Five steps of the in-checkpoint MTP head propose tokens, then one target pass verifies them. That pass runs AITER MXFP4 experts, supported gfx950 mHC and shape-scoped fused KDA, with reference fallbacks where required; expert execution keeps the model’s SwiGLU gate and clamp semantics. Graph replay removes repeated launch work, while KV and Mamba state must still point at the same visible committed prefix after a rejection or a stop-token truncation.

Figure 7

EAGLE 5/1/6: five drafts, one verification pass

  1. 01 Draft

    Five MTP steps

    The MTP head that ships inside the checkpoint proposes the next tokens, one step at a time.

  2. 02 Verify

    One target pass, six tokens

    The full MXFP4 model checks every proposal at once: DSA attention, AITER MXFP4 MoE, KDA and mHC on TP4/EP4.

  3. 03 Commit

    4.91 tokens per pass

    At c16, 78.14% of drafts were accepted across 1,595 completed requests.

Acceptance length is generated tokens per target-verification call, bonus token included; acceptance rate is accepted drafts over the five proposed. Both measured on the c16 AgentX rung. Source: model card, eagle-pipeline-mechanism.json

06Measured, not assumed: tuning KDA

Start from where verification time goes, then tune what the trace shows.

Inside target verification, the c16 capture records 535.5 ms of MoE, 112.6 ms of mHC and 91.4 ms of KDA device-event residence per rank. MoE is the largest; KDA is a smaller, concrete opportunity to cut repeated launches and work.

Figure 8

Target-verification device time per rank at c16

  • MoE
    535.5 ms
  • mHC
    112.6 ms
  • KDA
    91.4 ms
Device-event residence per rank, averaged across four ranks, from the c16 serving capture. Source: model card, final-c16-trace-summary.json

The KDA study ran on an isolated MI350X VF (gfx950) with synthetic GLM TP4-shaped tensors — an operator study, not a checkpoint comparison or a new MI355X serving run. The first case has eight requests with eight verification tokens each, and it climbs in three steps.

Figure 9

Three steps from two kernels to one faster kernel

  • 1 · Reference: two kernels
    38.9 μs

    Causal convolution, then gating and delta-rule recurrence.

  • 2 · First fusion, 4-value tile (BV=4)
    41.7 μs

    One kernel per replay, yet 7.2% slower: the Q/K convolution is recomputed for each of 32 value tiles. Fewer launches alone does not prove a speed-up.

  • 3 · Tuned fusion, 16-value tile (BV=16)
    28.4 μs

    Sharing that work across more values shrinks the value-tile grid from 32 to 8: 27.0% less time than the reference.

Paired graph timings from a separate, unprofiled microbenchmark: three reference / fused / fused / reference (ABBA) repeats, fixture clones excluded. These are operator measurements, not a percentage to add to the serving curves. Source: model card, kda-trace-optimization.json (batch 8, eight verification tokens)

Check the serving shape, keep the fallback

EAGLE 5/1/6 verifies six tokens, so the fused kernel had to win at that shape. The matched six-token sweep confirms the direction at seven batch sizes, and it also shows the gain shrinking as batches grow. That is why dispatch depends on shape: batches 1–2 keep the four-value tile, and the qualified larger batches use the 16-value tile.

Figure 10

Six-token verification across seven batch sizes

  • Batch 1 · BV=4 −48.3%
    Reference path: 25.22 μs
    Selected fused kernel: 13.04 μs
  • Batch 2 · BV=4 −41.8%
    Reference path: 26.19 μs
    Selected fused kernel: 15.24 μs
  • Batch 4 · BV=16 −29.4%
    Reference path: 28.26 μs
    Selected fused kernel: 19.95 μs
  • Batch 8 · BV=16 −26.9%
    Reference path: 31.50 μs
    Selected fused kernel: 23.02 μs
  • Batch 12 · BV=16 −16.5%
    Reference path: 34.06 μs
    Selected fused kernel: 28.44 μs
  • Batch 14 · BV=16 −13.1%
    Reference path: 35.11 μs
    Selected fused kernel: 30.50 μs
  • Batch 16 · BV=16 −12.3%
    Reference path: 37.66 μs
    Selected fused kernel: 33.03 μs
Reference path versus the fused KDA kernel the PR selects for each batch size (BV = value-tile width). The time saved falls from 48.3% at batch 1 to 12.3% at batch 16. Source: kda-trace-optimization.json · median of three paired ABBA arm means per shape

The PR reuses the fused KDA path only where it was qualified: gfx950, batches 1–16, six or eight verification tokens, 16 local heads, 128-wide states, BF16 activations, FP32 convolution weights, no convolution bias and the −5 gate bound. Every other ROCm shape keeps the reference path. For every measured shape, three changed-input graph replays matched exactly on active output and convolution buffers, and intermediate SSM states stayed within an absolute tolerance of 0.004.

07Two more measured choices

Not every gain needs a new kernel. Two came from picking the right existing path for GLM-5.3-Flash’s shapes.

mHC backend selection. GLM-5.3-Flash mixes four residual streams across 4,096 channels around its attention and MLP blocks. With AITER enabled on gfx950, the runtime selects AITER’s HIP kernels for this geometry: the same launch count and caller RMSNorm, shorter kernels, and the same numerical oracle passing on both routes.

Vision merge as a matrix multiply. The vision downsample has kernel size equal to stride (2×2), no padding and one group, so each patch is an independent linear projection. The existing Conv2dLayer handles that case as unfold + linear, bypassing MIOpen’s convolution search. On the model’s non-contiguous [256, 1024, 2, 2] merge layout, a layout copy and one bias-enabled matrix multiply replace the transpose / convolution / add sequence, and BF16 output passes the same numerical threshold.

Figure 11

Picking the path, measured per operator

  • mHC pre + RMSNorm −67.3%
    Reference path: 30.60 μs
    Selected path: 10.01 μs
  • mHC post −79.0%
    Reference path: 9.66 μs
    Selected path: 2.03 μs
  • Vision merge, warmed graph −59.5%
    Reference path: 63.28 μs
    Selected path: 25.62 μs
  • Vision merge, first call in a fresh process −26.5%
    Reference path: 4.501 s
    Selected path: 3.307 s

Each row is scaled to its own reference time.

Isolated MI355X operator measurements (mHC at 32 tokens), not percentages attributed to the AgentX serving curves. Source: model card, mhc-vision-operator-study.json

08MI355X next to supplied B300 estimates

A frontier illustration, not a benchmark win: one curve is measured, the other is estimates we were given.

The cyan curve is measured on four MI355X GPUs (gfx950 VFs) with this checkpoint. The dashed amber curve is supplied estimates for four B300 GPUs; we did not reproduce B300 runs or obtain their raw artifacts. The corpus, 256K context cap, GPU count and EAGLE shape align, but hardware, DSA backend and serving installations differ, so this is not a controlled cross-hardware speedup claim.

Figure 12

Measured MI355X frontier against supplied B300 estimates

Measured four-MI355X frontier versus supplied four-B300 estimates Aggregate output tokens per second against P90 output tokens per second per user, concurrency 1 to 48. The measured MI355X curve serves more aggregate output at every level; at c48 the B300 estimate has the higher per-user rate, 90.3 versus 73.0 tok/s. B300 · supplied estimate · c1 — Output tok/s: 219.0 · P90 tok/s/user: 349.2 B300 · supplied estimate · c2 — Output tok/s: 398.0 · P90 tok/s/user: 298.9 B300 · supplied estimate · c4 — Output tok/s: 695.0 · P90 tok/s/user: 264.6 B300 · supplied estimate · c8 — Output tok/s: 964.0 · P90 tok/s/user: 198.3 B300 · supplied estimate · c16 — Output tok/s: 1,446.0 · P90 tok/s/user: 148.0 B300 · supplied estimate · c32 — Output tok/s: 1,680.0 · P90 tok/s/user: 94.0 B300 · supplied estimate · c48 — Output tok/s: 1,721.0 · P90 tok/s/user: 90.3 MI355X · OneNexus MXFP4, measured · c1 — Output tok/s: 262.19 · P90 tok/s/user: 380.35 MI355X · OneNexus MXFP4, measured · c2 — Output tok/s: 441.34 · P90 tok/s/user: 337.36 MI355X · OneNexus MXFP4, measured · c4 — Output tok/s: 768.45 · P90 tok/s/user: 294.51 MI355X · OneNexus MXFP4, measured · c8 — Output tok/s: 1,269.90 · P90 tok/s/user: 238.61 MI355X · OneNexus MXFP4, measured · c16 — Output tok/s: 1,831.06 · P90 tok/s/user: 176.74 MI355X · OneNexus MXFP4, measured · c32 — Output tok/s: 2,175.11 · P90 tok/s/user: 106.55 MI355X · OneNexus MXFP4, measured · c48 — Output tok/s: 2,355.83 · P90 tok/s/user: 73.01 Measured four-MI355X frontier versus supplied four-B300 estimates Aggregate output tokens per second against P90 output tokens per second per user, concurrency 1 to 48. The measured MI355X curve serves more aggregate output at every level; at c48 the B300 estimate has the higher per-user rate, 90.3 versus 73.0 tok/s. B300 · supplied estimate · c1 — Output tok/s: 219.0 · P90 tok/s/user: 349.2 B300 · supplied estimate · c2 — Output tok/s: 398.0 · P90 tok/s/user: 298.9 B300 · supplied estimate · c4 — Output tok/s: 695.0 · P90 tok/s/user: 264.6 B300 · supplied estimate · c8 — Output tok/s: 964.0 · P90 tok/s/user: 198.3 B300 · supplied estimate · c16 — Output tok/s: 1,446.0 · P90 tok/s/user: 148.0 B300 · supplied estimate · c32 — Output tok/s: 1,680.0 · P90 tok/s/user: 94.0 B300 · supplied estimate · c48 — Output tok/s: 1,721.0 · P90 tok/s/user: 90.3 MI355X · OneNexus MXFP4, measured · c1 — Output tok/s: 262.19 · P90 tok/s/user: 380.35 MI355X · OneNexus MXFP4, measured · c2 — Output tok/s: 441.34 · P90 tok/s/user: 337.36 MI355X · OneNexus MXFP4, measured · c4 — Output tok/s: 768.45 · P90 tok/s/user: 294.51 MI355X · OneNexus MXFP4, measured · c8 — Output tok/s: 1,269.90 · P90 tok/s/user: 238.61 MI355X · OneNexus MXFP4, measured · c16 — Output tok/s: 1,831.06 · P90 tok/s/user: 176.74 MI355X · OneNexus MXFP4, measured · c32 — Output tok/s: 2,175.11 · P90 tok/s/user: 106.55 MI355X · OneNexus MXFP4, measured · c48 — Output tok/s: 2,355.83 · P90 tok/s/user: 73.01
Solid: measured on four MI355X GPUs. Dashed with hollow points: supplied estimates for four B300 GPUs, not reproduced by us. The measured MI355X c64 point is dominated by c48 and is left out. Source: model card, mxfp4-agentx-frontier.json and b300-agentx-estimates.json

At c48, the measured MI355X arm has a 36.9% higher aggregate output rate but a 19.1% lower P90 per-user rate than the supplied B300 estimate. Across eight MI355X rungs, 11,810 requests completed with zero errors.

ConcurrencyMI355X output tok/s (measured)MI355X P90 tok/s/user (measured)B300 output tok/s (estimate)B300 P90 tok/s/user (estimate)
1262.19380.35219349.2
2441.34337.36398298.9
4768.45294.51695264.6
81,269.90238.61964198.3
161,831.06176.741,446148.0
322,175.11106.551,68094.0
482,355.8373.011,72190.3

09Reproduce it

Every figure in this post traces back to a JSON summary and a protocol in the model repository.

The serving recipe pins the checkpoint revision, container, source, tokenizer and launch arguments. The AgentX measurements use SGLang source 5fd4387a753cdc88c0c87efb1bcabab1efabfc93, and the qualification record identifies the source used for serving qualification separately. Quality runs use BF16 KV with graphs and speculation disabled. Raw restricted prompts are not redistributed.

References

Model card: OneNexus/GLM-5.3-Flash-MXFP4, SGLang PR #39273, quality protocol, FP8 / MXFP4 serving protocol, KDA trace study, mHC and vision operator study, AMD reference checkpoint.