Skip to content

release: 0.11.0, Solr 10.1, incrased robustness and indexing performance - #525

Merged
jbaiter merged 1 commit into
mainfrom
release-0.11.0
Nov 16, 2026
Merged

jbaiter merged 1 commit into
mainfrom
release-0.11.0

Conversation

@jbaiter

@jbaiter jbaiter commented Nov 16, 2026

Copy link
Copy Markdown
Member

Changed

  • Added compatibility with Solr 10.1

Fixed

  • Byte offsets of external OCR files are now correct across source-pointer region boundaries. A region that ends inside a multi-byte character used to lose or duplicate text and shift every following offset; the truncated character is now indexed as U+FFFD. Offsets after supplementary characters no longer drift, and a small read that splits a surrogate pair no longer fails indexing. Reindex fields that use ExternalUtf8ContentFilter for the corrected offsets to take effect, existing indexes keep the offsets they were built with. As a nice side effect, this brings a 4-10% performance boost for indexing, depending on the OCR.

**Changed**
- Added compatibility with Solr 10.1

**Fixed**
- Byte offsets of external OCR files are now correct across source-pointer region
  boundaries. A region that ends inside a multi-byte character used to lose or
  duplicate text and shift every following offset; the truncated character is now
  indexed as `U+FFFD`. Offsets after supplementary characters no longer drift, and
  a small read that splits a surrogate pair no longer fails indexing.
  **Reindex fields that use `ExternalUtf8ContentFilter`** for the corrected
  offsets to take effect, existing indexes keep the offsets they were built with.
  As a nice side effect, this brings a 4-10% performance boost for indexing,
  depending on the OCR.
@jbaiter
jbaiter merged commit 63c34ce into main Nov 16, 2026
7 checks passed
@jbaiter
jbaiter deleted the release-0.11.0 branch November 16, 2026 13:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants