Skip to content

fix: handle empty masks in last-token pooling - #858

Open
elysiopires wants to merge 2 commits into
qdrant:mainfrom
elysiopires:codex/fix-empty-mask-last-token
Open

elysiopires wants to merge 2 commits into
qdrant:mainfrom
elysiopires:codex/fix-empty-mask-last-token

Conversation

@elysiopires

Copy link
Copy Markdown

Last-token pooling returns a padding embedding when an entire attention-mask row is zero. With a zero-length sequence it raises an argmax error. This can give a misleading nonzero result for a custom model or tokenizer that returns no unmasked tokens.

Return a zero vector for these two cases, consistent with mean pooling on fully masked input. Keep the embedding dtype and preserve the existing behavior for rows with real tokens, including left and right padding.

Validation:

  • The two new cases failed before the change and passed afterward.
  • PYTHONPATH=. python -m pytest tests/test_common.py -q --tb=short: 9 passed.
  • Focused last-token and custom-model pooling tests: 5 passed.
  • Ruff E/F checks and formatting check passed on the changed files.

These tests use synthetic model outputs; no external model inference was run.

@elysiopires
elysiopires requested a review from joein as a code owner October 31, 2026 15:36
@coderabbitai

coderabbitai Bot commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

last_token_pooling now returns zeros for empty sequences and fully masked rows. For other rows, it retains last-unmasked-token selection. Tests cover these behaviors and verify output shape and dtype.

Estimated code review effort: 2 (Simple) | ~8 minutes

Change: Bug fix

Possibly related PRs

  • qdrant/fastembed#874: Adds models to LastTokenNormalizedEmbedding, which uses the pooling behavior changed here.

Suggested labels: risk:medium

Merge Risk: Minimal · up to 11989

This change makes last-token pooling return zero vectors for empty or fully masked inputs instead of a padding embedding or an error. Behavior for rows with real tokens is unchanged, and tests cover the new cases. No actionable merge-blocking risk was identified.

Pre-merge checks: Passed 5

Check name Status Explanation
Title check Passed The title clearly and concisely describes the main change: handling empty masks in last-token pooling.
Description check Passed The description directly explains the affected behavior, intended fix, preserved behavior, and validation for the changeset.
Docstring Coverage Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentat…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 1 functions across 2 files.

Finishing Touches

Generate unit tests (beta)

  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 0c957b5a-6353-4025-9946-ae9a92d94833
Commits

Reviewing files that changed from the base of the PR and between 2518f59 and 1198967.

Files selected for processing: 1

  • fastembed/common/utils.py
Files skipped from review as they are similar to previous changes: 1

  • fastembed/common/utils.py

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.

@chrikrah chrikrah left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@elysiopires from my side this is ready to merge at 2518f59. Both new tests fail on the base and pass here.

$ .venv/bin/python -m pytest -q -p no:cacheprovider tests/test_common.py
9 passed in 0.54s                     # head 2518f59, CPython 3.12
# fastembed/common/utils.py from f91d916 (base), your tests kept
FAILED tests/test_common.py::test_last_token_pooling_with_fully_masked_sequences
FAILED tests/test_common.py::test_last_token_pooling_with_zero_length_sequences
2 failed, 7 passed in 0.48s

$ python -I probe_qwen_tok.py   # load_tokenizer() on Qwen/Qwen3-Embedding-0.6B, encode_batch(["", "hello world"])
[[1, 0, 0], [1, 1, 1]]
[['<|endoftext|>', '<|endoftext|>', '<|endoftext|>'], ['hello', 'Ġworld', '<|endoftext|>']]
# not run: an ONNX model end to end

non-blocking: zeros or a ValueError, like the other _pool guards, are both defensible. I'd keep zeros so last-token matches mean_pooling on the same row, at the cost of a silent zero vector in the index. Only a custom tokenizer with no EOS gets here; Qwen3 appends <|endoftext|> even to "".

Worth a one-line comment at utils.py:56 naming that case?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants