Skip to content

[Feature Request]: HNSW quantization, narrow vector columns, exact prefiltered search, memory-sized models #31467

Open

Description

@cpegeric

Is there an existing issue for the same feature request?

  • I have checked the existing issues.

Is your feature request related to a problem?

1. An HNSW index stores every vector at the width of its column, with no way to trade
   precision for a smaller graph (IVF-Flat and IVF-PQ already accept QUANTIZATION).
2. HNSW only indexes vecf32 / vecf64; vecf16, vecbf16, vecint8 and vecuint8 columns cannot be indexed.
3. A prefiltered search over few qualifying rows walks the graph and can come back short.
4. Each model holds at most max_index_capacity rows (default 1M) whatever the element width, so a
   large index is split into many models; every query searches all of them in full (5 models on
   5M rows measured ~5x slower).

Describe the feature you'd like

  • QUANTIZATION on HNSW (float32, float16, bf16, float64, int8, uint8), int8/uint8 with one trained range per index.
  • Index vecf16 / vecbf16 / vecint8 / vecuint8 columns in their own element type.
  • Exact prefiltered search when the qualifying row count is at most hnsw_prefilter_exact_threshold.
  • max_index_capacity default 0: size each model from available host memory (as IVF-PQ), so a quantized index fits one model.

Describe implementation you've considered

Design doc: docs/design/20261026-hnsw-quantization.md. The int8/uint8 range lives in a key/value hidden table (hnsw_dict, no upgrade step); hnsw_create(params, cfg) counts, samples and scans the source itself (no CROSS APPLY).

Documentation, Adoption, Use Case, Migration Strategy

CREATE INDEX idx USING hnsw ON t(v) op_type 'vector_l2_ops' quantization 'int8';
Existing indexes keep working unchanged; they now size new models from memory unless max_index_capacity is set.

Additional information

Follow-up (separate PR): CDC tail + idxcron merge, so syncing a single large model does not rewrite it on every iteration.

Activity

  1. MatrixAdventurer commented on Oct 27, 2026

    @MatrixAdventurer
    Contributor

    分级建议:不适用缺陷严重级(kind/feature)。能力增强类(HNSW 量化、vecf16/vecbf16/vecint8/vecuint8 索引、预过滤精确召回、按内存定容);其中"5M 行 5 模型全量扫描实测约 5x 慢"若需要单列可按 performance 口径参考 severity/s2。模块/目录:向量索引 HNSW(量化、模型容量、预过滤阈值),设计文档 docs/design/20261026-hnsw-quantization.md 已就绪。疑似重复:未发现同类 feature 请求;相关 HNSW 问题 #31456(BVT 缓存陈旧)在跟踪中。建议负责人:cpegeric(作者,含设计文档与 HNSW 测试背景)。本操作由LW授权的MOI智能体提交

  2. XuPeng-SH commented on Oct 28, 2026

    @XuPeng-SH
    Contributor

    QA status for merged main f847803802289b2b2d0df4f0a2f842c2804b823a (#31468): the repository's new usearch native dependency was built with make -j1 cgo NATIVE_BUILD_JOBS=1 and its native provenance verification passed. This confirms dependency build/provenance only; it does not establish HNSW feature correctness.

    The current environment does not have enough headroom under the existing resource guard for the affected HNSW test link or a current-head service build. Quantization and narrow-column SQL, trained ranges and extreme values, exact prefilter results, model capacity, save/load, incremental updates/deletes and recall remain unverified. No performance conclusion is made.

    Acceptance is also blocked by the separate reporter's live findings in #31493, not reproduced in this environment: on f847803 in an isolated one-CN cluster, a committed float16-out-of-range vector was followed by a valid row, but indexed results remained incomplete through 30 seconds in 3/3 fresh databases while exact scans and valid-only controls returned the expected rows. The retry/watermark cause is not confirmed. Insert and update paths and consistency/recovery after the error still need verification.

    Keeping the existing assignee pending acceptance.

  3. slyunnn commented on Nov 4, 2026

    @slyunnn
    Collaborator

    Feature verification for #31467 — preliminary results (2026-11-04)

    Status: incomplete; no feature acceptance conclusion yet.

    The test used the exact-commit source archive for MatrixOne 2b16c3abc620a02cacd081f27ef9c14b2cc335ef; GitHub compare shows feature merge f847803802289b2b2d0df4f0a2f842c2804b823a (#31468) is an ancestor. The normal Git fetch failed, so this was an API archive rather than a Git checkout. The built binary reported 2b16c3abc6 and its SHA256 was 0d75f6a906d16c738f4993eead91a8dc5928332e32379086e1e6d7ea3d7cec58.

    The repository vector_hnsw_quantization.sql was executed with mo-tester through Proxy and two CN connections. With JDBC metadata comparison disabled, each run reported 186/186 successful and zero ignored. The initial run with metadata comparison enabled reported 183/186: three failures returned expected numeric values but had different JDBC type/length metadata for two system variables. The discrepancy remains unclassified; the initial run was not a full golden comparison pass.

    A separate three-row int8 quantized HNSW index was queried, the service was restarted with the same test data, then queried through another CN. The post-restart plan showed Vector Index Scan, and indexed and exact Top-3 both returned 1,2,3. The test database was dropped.

    The earlier #31493 issue was fixed and independently verified by its owner; that historical result was not reproduced in this run. No new reproducible product bug was confirmed. Large-scale recall, actual memory-based model sizing under multiple limits, concurrency and broader recovery remain unverified. No new regression PR was created. Further testing is needed before acceptance.

  4. slyunnn commented on Nov 11, 2026

    @slyunnn
    Collaborator

    TKE follow-up status (2026-11-11)

    • The million-row HNSW quantization run exposed repeatable zero recall for bf16 and int8 on a correlated synthetic dataset; tracked in #32513. Same-query float32 controls and exact baselines are included there. The earlier recall workflow's green status only indicates its plan/result-count checks passed; it did not assert recall.
    • In the first 6 GiB capacity run, one bulk insert exceeded its hash-build memory budget before index creation. Batched inserts subsequently loaded all 1M rows and the auto-capacity index produced two models. The follow-up test failed because metadata nrow was zero; the tested version documents zero as an unknown sentinel, not a zero-row model. The test was corrected to report that evidence limitation, and another 6 GiB run is in progress.
    • The above observations do not complete feature acceptance. The corrected capacity run, higher-memory comparison, concurrency and broader recovery remain pending.
  5. slyunnn commented on Nov 11, 2026

    @slyunnn
    Collaborator

    TKE 6 GiB capacity follow-up

    Run 42955788991 completed successfully and cleaned its test database and cluster. It verified the actual CN cgroup limit was 6 GiB, inserted and counted 1,000,000 vectors of dimension 1024, and observed two auto-sized HNSW models with nrow 657,458 and 342,542. With MAX_INDEX_CAPACITY 200000, it observed five models of 200,000 rows each. The tests verify this specific configuration; they do not establish all memory-budget cases.

    A distinct single-query recall@3 of zero on the explicit-capacity float32 index was recorded separately in #32513; the capacity workflow does not treat approximate-query equality as its acceptance criterion.

    The matching 12 GiB comparison is queued behind another shared Nightly run. Until it runs, the effect of changing the memory limit remains unverified.

  6. slyunnn commented on Nov 11, 2026

    @slyunnn
    Collaborator

    12 GiB comparison status

    Run 42957964460 failed during cluster readiness, before any feature SQL ran. Both AP CN Pods remained Pending; Kubernetes scheduling events reported insufficient CPU/memory and node placement constraints. The run-owned cleanup completed successfully.

    The 12 GiB capacity comparison therefore remains unverified. The successful 6 GiB results reported above remain valid for that configuration only; they do not establish the effect of changing memory limits. The independently reproduced recall issue remains tracked in #32513.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions