2026-11-10T03:43:03.248556 image/svg+xml Matplotlib v3.11.2, https://matplotlib.org/ GS1-0.8B GS1-2B GS1-4B 0 20 40 60 80 Complete decision latency (ms) 54.1 53.9 72.9 54.6 54.3 73.8 12.5 18.6 35.2 256-token budget · 1 question GS1-0.8B GS1-2B GS1-4B 0 25 50 75 100 125 Complete decision latency (ms) 56.2 57.7 105.2 56.9 58.1 118.5 23.6 42.3 86.3 1,024-token budget · 1 question GS1-0.8B GS1-2B GS1-4B 0 200 400 600 Complete decision latency (ms) 422.9 422.7 566.2 436.5 434.1 589.2 70.0 121.2 240.4 256-token budget · 8 questions GS1-0.8B GS1-2B GS1-4B 0 200 400 600 800 1000 Complete decision latency (ms) 436.8 443.5 673.4 450.3 459.8 939.5 178.4 347.9 754.3 1,024-token budget · 8 questions GS1 inference latency · current release RTX 4090 (48 GiB reported) · BF16 · 5 warmups + 30 calls per group Bars: P50; whiskers: P95 · includes compilation + probabilities; excludes load, capture and HTTP GS1 Default GS1 eager GS1 CUDA Graph