Repository navigation
Performance improvements - #137
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR makes the entire write path O(n log n) for indexed bulk saves (and O(n) when no B-tree index is involved), with no functionality removed and a
Contextvalue-semantics bug fixed along the way.Typical graph saves are ~5–8× faster; index-heavy bulk loads are ~16–21× faster at 5k entities and the gap widens without bound with dataset size.
Results
Relation-graph saves (at n = 1000 entities)
Index heavy write costs (at n = 5000 entities)
@FullTextIndex@Unique@HashIndexThis is how the gains compound with scale. Before was O(n) per insert (quadratic in bulk); after is per-insert O(1) or O(log n). So the multiplier grows with the dataset:
The optimized path stays near-flat per entity (~5 µs/save storage, ~28 µs/save full-text) at both 5k and 20k, while the baseline grows quadratically — so the multiplier keeps widening with the dataset.
Reads also improved
The typed
Storage<T>boxes +ObjectIdentifierkeys also speed up entity resolution (find/findAll/all), so reads that materialize many entities got faster too. Narrow indexed lookups (a handful of ids) were already sub-millisecond and are unchanged..filter)(Millisecond-scale timings at high RSD, so treat these as directional. The reliable signal: resolve-dominated reads — full scans that materialize every entity — get ~2.3–2.5× from the typed-storage/
ObjectIdentifierchanges; the indexed sort resolves the full set too but its B-tree traversal dilutes the win, and index-served narrow reads were already sub-millisecond and stay flat.)Why it was slow
Every index update did
resolve → mutate → save, and the storage did copy-out / mutate / copy-back. Both leave the backing collection shared at mutation time, forcing a full O(n) copy-on-write on every insert — masquerading as O(1). The same pattern lived in five places:EntitiesRepository,Index.HashableValue,Index.ComparableValue, theUnique.*indexes, andFullTextIndex.HashableValue. On top of that,.nonerelations were writing emptyLinknodes that did nothing, theupdatedAtmetadata wrote two indexes per save when only one is ever read, and every storage access resolved its key by reflection (String(reflecting:)).What changed
Per-insert cost now follows the index's data structure:
@HashIndex,@FullTextIndex, Hashable-only@UniqueMap)@Index, Comparable@Unique, and the always-onupdatedAtmetadata indexThe
log nis BTree depth. The only universal contributor isupdatedAtmetadata (one B-tree write per save by default) — so with metadata disabled and hash-only indexes the path is genuinely O(n).relations write optimisationLinkwrites for.nonerelationsentity repository CoW optimizationsubscript(_:default:)) — O(n²)→O(1)/insertnon copying mutation helpersContext.mutate/mutateIfPresent— take unique ownership, mutate in placemigrated all indexes to structstyped storageStorage<T>boxes — oneas?cast per lookup instead of per elementObjectIdentifier keysremoved odd hashable metadata indexupdatedAtwrites only the B-tree (for sort/range); dropped the redundant, never-read hash indexTests
Value-type indexes fixed a latent
Contextvalue-semantics bug: index storage was reference-shared, so copying aContextand mutating one copy silently corrupted the other's indexes. Now copies are independent (verified by test). All 382 tests pass, including the index / collision / full-text / value-semantics suites; reads still hit every index (indexed filters and sorts remain faster than scans).