Skip to content

hmx_runtime: pin the bus (DDR) corner along with the core corner (PLaMo decode 1.56 -> 1.26 s/token on V69) - #16

Draft
take-cheeze wants to merge 1 commit into
bevformer-onnx-remotefrom
plamo-bus-corner
Draft

take-cheeze wants to merge 1 commit into
bevformer-onnx-remotefrom
plamo-bus-corner

Conversation

@take-cheeze

Copy link
Copy Markdown
Member

hmx_rt_power voted the DSP core corner to TURBO but left the bus/DDR clock to DCVS.

On the Xiaomi 12S (Snapdragon 8+ Gen 1, V69), PLaMo 3 Translate 2B decode (six 4-layer HMX row-block programs through onnx-remote) varied from run to run. In 8 of 9 runs, a random 1–4 of the six programs took ~244 ms instead of ~185 ms. Which ones changed with each run and with load order.

Ruled out with on-device diagnostics (diagnostic skel variants, not part of this PR):

  • VTCM fallback: the VTCM regions were granted on every run, acquired in ~33 µs.
  • Allocation placement: allocating the intermediates as the weight block's tail changed nothing.
  • Core clock: HAP_perf_get_pcycles/µs read 1401 MHz on every run, slow ones included.

Per-call profile of a slow program: activation-sized HVX kernels were 10–100× slower and the HMX GEMMs ~4×, while the int4 weight dequant was unchanged. That pattern fits latency-bound DDR traffic.

Change: after the core vote, also call HAP_power_set_bus_corner(TURBO, TURBO, TURBO) and HAP_power_set_sleep_latency(100).

Result (same tokens throughout):

before after
per-program device time 185–245 ms, varies per run 184–186 ms on every program, 9 runs
greedy decode, 41-token prompt 1.63 s/token 1.26 s/token
greedy decode, 91-token prompt 1.56 s/token 1.27 s/token
prompt-lookup speculative decode, 91-token prompt 1.15 s/token 0.89 s/token

The driver and test that go with this are in the onnxsim PR (link to follow).

🤖 Generated with Claude Code

https://claude.ai/code/session_01EqPDeP1iLb8YA1xe9wneS5

…e core corner

hmx_rt_power voted the DSP core corner to TURBO but left the bus clock to DCVS.
On the Xiaomi 12S (V69) that made PLaMo 3 decode vary run to run: a random 1-4 of
the six layer-group programs took ~244 ms instead of ~185 ms, at the same
1401 MHz core clock and with VTCM granted on every run. The slowdown was in the
latency-bound kernels (activation-sized HVX loops 10-100x, HMX GEMMs ~4x), not
the weight dequant.

With HAP_power_set_bus_corner(TURBO) and a 100 us sleep latency, every program
ran at 184-186 ms in every run (9 runs), with the same tokens:
greedy decode 1.56-1.63 -> 1.26 s/token.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EqPDeP1iLb8YA1xe9wneS5
@take-cheeze
take-cheeze marked this pull request as draft November 10, 2026 00:23
@take-cheeze

Copy link
Copy Markdown
Member Author

Correction: the speedup did not reproduce the next morning. Marking this draft.

The bus-vote artifacts that ran all six programs at 185 ms in 9 consecutive runs on 2026-11-09 evening were slow again on 2026-11-10 morning. Five or six of the six programs ran at ~244 ms in every run (greedy decode 1.53 s/token, prompt-lookup speculative decode 1.59 s/token on the 41-token prompt), with the same binaries. So the evening's result was a time-dependent effect, not this change.

New facts so far:

  • Within one run, the slow set is fixed for every pass, so it is decided at load or first-run time.
  • The pattern follows system state over time, not the code.

I'm now checking whether the DSP heap's physical pages are fragmented (counting 4 KB page discontinuities over each program's weight and intermediate blocks). Small pages would mean TLB misses in the latency-bound kernels. I'll update this PR once there's a cause that holds across days. The bus vote may still be worth keeping, but it is not the fix described above.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant