Repository navigation
hmx_runtime: pin the bus (DDR) corner along with the core corner (PLaMo decode 1.56 -> 1.26 s/token on V69) - #16
take-cheeze wants to merge 1 commit into
Conversation
…e core corner hmx_rt_power voted the DSP core corner to TURBO but left the bus clock to DCVS. On the Xiaomi 12S (V69) that made PLaMo 3 decode vary run to run: a random 1-4 of the six layer-group programs took ~244 ms instead of ~185 ms, at the same 1401 MHz core clock and with VTCM granted on every run. The slowdown was in the latency-bound kernels (activation-sized HVX loops 10-100x, HMX GEMMs ~4x), not the weight dequant. With HAP_power_set_bus_corner(TURBO) and a 100 us sleep latency, every program ran at 184-186 ms in every run (9 runs), with the same tokens: greedy decode 1.56-1.63 -> 1.26 s/token. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EqPDeP1iLb8YA1xe9wneS5
|
Correction: the speedup did not reproduce the next morning. Marking this draft. The bus-vote artifacts that ran all six programs at 185 ms in 9 consecutive runs on 2026-11-09 evening were slow again on 2026-11-10 morning. Five or six of the six programs ran at ~244 ms in every run (greedy decode 1.53 s/token, prompt-lookup speculative decode 1.59 s/token on the 41-token prompt), with the same binaries. So the evening's result was a time-dependent effect, not this change. New facts so far:
I'm now checking whether the DSP heap's physical pages are fragmented (counting 4 KB page discontinuities over each program's weight and intermediate blocks). Small pages would mean TLB misses in the latency-bound kernels. I'll update this PR once there's a cause that holds across days. The bus vote may still be worth keeping, but it is not the fix described above. |
hmx_rt_powervoted the DSP core corner to TURBO but left the bus/DDR clock to DCVS.On the Xiaomi 12S (Snapdragon 8+ Gen 1, V69), PLaMo 3 Translate 2B decode (six 4-layer HMX row-block programs through onnx-remote) varied from run to run. In 8 of 9 runs, a random 1–4 of the six programs took ~244 ms instead of ~185 ms. Which ones changed with each run and with load order.
Ruled out with on-device diagnostics (diagnostic skel variants, not part of this PR):
HAP_perf_get_pcycles/µs read 1401 MHz on every run, slow ones included.Per-call profile of a slow program: activation-sized HVX kernels were 10–100× slower and the HMX GEMMs ~4×, while the int4 weight dequant was unchanged. That pattern fits latency-bound DDR traffic.
Change: after the core vote, also call
HAP_power_set_bus_corner(TURBO, TURBO, TURBO)andHAP_power_set_sleep_latency(100).Result (same tokens throughout):
The driver and test that go with this are in the onnxsim PR (link to follow).
🤖 Generated with Claude Code
https://claude.ai/code/session_01EqPDeP1iLb8YA1xe9wneS5