Skip to content

fix(index): exclude children of null lists from LabelList postings - #11283

Open
ddupg wants to merge 2 commits into
lance-format:mainfrom
ddupg:fix/label-list-null-children
Open

ddupg wants to merge 2 commits into
lance-format:mainfrom
ddupg:fix/label-list-null-children

Conversation

@ddupg

@ddupg ddupg commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

Split out from #11273.

Arrow permits a null list to span a non-empty range in the child array. LabelList used the offsets alone when flattening lists, so a label stored under a null parent could acquire a posting for that row.

Build and update now read each parent row's address once during list expansion. A valid list pairs its labels with that address. A null list contributes its address to the list-null bitmap and emits no postings. The expansion reuses the sliced child array when there are no hidden values to remove.

The regression test inspects the stored index for both List and LargeList. It checks that a hidden label has no posting, the null list is still recorded, and a null element inside a valid list retains its posting.

All 23 LabelList unit tests passed locally, including both new regression cases. Formatting passed. Workspace Clippy on macOS arm64 stopped in unchanged byte_stream_split.rs at lines 358 and 393 (unreachable expressions after the NEON dispatch).

@github-actions github-actions Bot added A-index Vector index, linalg, tokenizer bug Something isn't working labels Nov 20, 2026
@ddupg
ddupg marked this pull request as ready for review November 20, 2026 02:24

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

lance-gatekeeper[bot]

This comment was marked as outdated.

@lance-gatekeeper lance-gatekeeper Bot added K-approved Latest Gatekeeper recommendation permits acceptance. and removed K-approved Latest Gatekeeper recommendation permits acceptance. labels Nov 20, 2026

@lance-gatekeeper lance-gatekeeper Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Gate recommendation: approve.

The combined expansion fixes null-parent postings while preserving null elements, list-level null tracking, and the stored index format.

@lance-gatekeeper lance-gatekeeper Bot added the K-approved Latest Gatekeeper recommendation permits acceptance. label Nov 20, 2026
@ddupg
ddupg force-pushed the fix/label-list-null-children branch from 0ef0112 to 703b098 Compare November 20, 2026 03:28

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-index Vector index, linalg, tokenizer bug Something isn't working K-approved Latest Gatekeeper recommendation permits acceptance.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant