Skip to content

Price-aware routing in the mesh head (Phase 1 market signal) - #1

Merged
xzyaoi merged 2 commits into
mainfrom
price-aware-routing
Sep 14, 2026
Merged

xzyaoi merged 2 commits into
mainfrom
price-aware-routing

Conversation

@xzyaoi

@xzyaoi xzyaoi commented Sep 14, 2026 •

Copy link
Copy Markdown

Implements the routing side of Phase 1 (price-aware peer selection). API half: opentela-ai/api.opentela.ai#7.

Part 1 — the gate's price order now reaches routing (Closes opentela-ai/api.opentela.ai#3)

X-Otela-Allowed-Peers remains the hard affordability constraint; within it, selection is no longer uniform:

  • src/internal/server/proxy_handler.go:
    • parseAllowedPeerOrder captures the billing gate's cheapest-affordable-first peer order from the header.
    • orderCandidatesByPrice re-orders routing candidates to follow that order; priceAwareScores turns it into exponentially decaying selection weights (rank r weighs decay^r) — with 0.5 and three affordable peers the cheapest wins ~57% of first picks while the rest still serve.
    • Order-preserving retry exclusion: a failed attempt deterministically prefers the next-cheapest allowed peer instead of a random one.
  • src/entry/cmd/root.go: routing.price_weight_decay (env OF_ROUTING_PRICE_WEIGHT_DECAY, default 0 = legacy uniform policies) — the policy is a tunable knob, not hardcoded.
  • docs/content/docs/tutorial/routing.mdx: documents the knob and the market-signal semantics.
  • Tests: src/internal/server/allowed_peers_test.go (order parsing, re-ordering, decay weighting, retry preference).

Part 2 — account sign-in produces a usable JWT (provider onboarding support; no tracked issue)

src/internal/account/link.go SignInEmail hardened to match how Better Auth actually issues JWTs:

  • Two-factor accounts answer sign-in 200 without a session — now detected and reported with actionable guidance (--token path).
  • Sign-in only sets the HttpOnly session cookie; the jwt plugin attaches set-auth-jwt only to GET /get-session. SignInEmail now keeps the cookie jar and follows up with an authenticated /get-session call to extract the usable JWT.
  • Response-body/close error handling hardened.
  • Tests: src/internal/account/link_test.go, including the sign-in → /get-session → JWT round-trip against an in-process auth server.

Note: this ships the account library path; wiring it into the provider boot flow is a small follow-up once the seller-side console flow lands.

Docs: dual-rail warning (related: opentela-ai/api.opentela.ai#4)

docs/content/docs/tutorial/settlement.mdx gains a prominent callout documenting the two independent billing models (legacy rates.yaml per-1000 direct-SPL settlement vs the API market's per-1M asks and credit ledger), the double-billing hazard of running both on the same traffic, and "deploy exactly one model per traffic path". Retiring/unifying the legacy rail is tracked in opentela-ai/api.opentela.ai#4 and not addressed here.

Verification

The billing gate stamps X-Otela-Allowed-Peers cheapest-affordable-first;
the head now lets that order influence selection instead of load-balancing
the affordable set uniformly:

- parseAllowedPeerOrder captures the gate's price order.
- routing.price_weight_decay (env OF_ROUTING_PRICE_WEIGHT_DECAY, default 0
  = legacy uniform policies) turns the order into exponentially decaying
  selection weights (rank r weighs decay^r): with 0.5 and three affordable
  peers the cheapest wins ~57% of first picks while the rest still serve.
- orderCandidatesByPrice + the order-preserving retry exclusion make a
  failed attempt deterministically prefer the next-cheapest allowed peer.
- X-Otela-Allowed-Peers remains the hard affordability constraint; the new
  signal only reorders/weights within it.

Docs: routing.mdx gains a "Price-aware routing (billing market)" section;
settlement.mdx gains an explicit do-not-combine warning separating the
legacy peer-to-peer settlement rail (rates.yaml, per-1000, direct SPL
transfers) from the api.opentela.ai market (per-1M asks, credit accounts),
which double-bill if both run on the same traffic.

Tests: 9 new server tests (order parsing, decay weights, tail weights,
cheapest-dominance simulation, cheapest-first ordering, stable unknown
peers, next-cheapest retry); full internal/server suite green.

Tracks api.opentela.ai#3 and api.opentela.ai#4 (this repo has issues
disabled; the API repo owns the billing design).
Better Auth's jwt plugin attaches the set-auth-jwt header only to
GET /get-session (after-hook matcher: context.path === "/get-session").
Sign-in/email merely sets the HttpOnly session cookie and its body offers
no guaranteed-JWT field, so a successful sign-in must be followed by an
authenticated /get-session call carrying that cookie; the usable JWT then
travels in that response's set-auth-jwt header.
Copilot AI lite review requested due to automatic review settings September 14, 2026 13:49

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@xzyaoi xzyaoi left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review (full-diff pass)

Verdict: approvable. Verified locally against current main (post-eth-easl#64 rebase): go build ./..., go vet, and go test ./internal/server/ ./internal/account/ green. Note this fork has Actions disabled, so these local runs are the verification of record.

Verified correct

  • Price-aware routing (closes opentela-ai/api.opentela.ai#3): switch precedence is right — priceAware (when 0 < price_weight_decay < 1 and the header is present) overrides scalability.weighted_routing, and absent header / decay 0 keeps byte-for-byte legacy behavior (wire-compatible for existing callers). X-Otela-Allowed-Peers remains the hard constraint; the signal only reorders/weights within it.
  • Retry semantics: orderCandidatesByPrice + order-preserving excludePeers make remaining[0] after a failed attempt deterministically the next-cheapest surviving allowed peer — exactly the issue's "prefer the next-cheapest allowed peer" ask.
  • priceAwareScores: decay^rank weighting with defensive tail weight for peers absent from the order; empty-input guard.
  • SignInEmail (account): matches Better Auth's actual JWT semantics — sign-in only sets the HttpOnly session cookie, and the jwt plugin attaches set-auth-jwt only on GET /get-session; the cookie-jar follow-up call is the correct fix. 2FA-redirect detection gives actionable guidance. Response close/error handling hardened.
  • Docs: routing.mdx documents the knob; the settlement.mdx dual-rail callout is accurate about the double-billing hazard and does not overstate (unification explicitly left to opentela-ai/api.opentela.ai#4).

Non-blocking nits (follow-ups)

  1. When routing.price_weight_decay and scalability.weighted_routing are both enabled, price-aware silently wins — worth one sentence in routing.mdx stating the precedence.
  2. No metrics on price-aware selection (e.g. picks-by-rank histogram) — would help operators tune the decay knob.
  3. Account library ships without a call site in entry/ — boot wiring is a stated follow-up; fine to merge as-is.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Price-aware routing in the mesh head (Phase 1): the affordable set is load-balanced uniformly, so asks don't influence which peer serves

2 participants