Skip to content

rpc: serve ListObjectVersions from the fullnode within its retention window - #28645

Draft
nickvikeras wants to merge 1 commit into
nickv/list-object-versionsfrom
nickv/list-object-versions-fullnode
Draft

nickvikeras wants to merge 1 commit into
nickv/list-object-versionsfrom
nickv/list-object-versions-fullnode

Conversation

@nickvikeras

@nickvikeras nickvikeras commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Description

Fullnodes keep every object version within their retention window, so they can serve ListObjectVersions themselves for objects whose history is retained, instead of answering UNIMPLEMENTED and leaving every listing to archival kv-rpc.

Mini design doc

Design. The fullnode handler reuses the shared request handling from #28632: ObjectVersionsQuery parses the request, bounding_checkpoint and plan fold the version range, cursors and at_checkpoint into one inclusive range and choose the QueryEnd reason, and frames emits the frames from ObjectVersionItems, each removal with the fields removal_read_mask selects. The handler adds the reads, the retention floor and, when the mask selects it, each removal's transaction. Before anything is listed, a blocking task reads the object's version at the bounding checkpoint and the retention floor. The items are then read in chunks of blocking RocksDB reads on ChunkedScan (up to chunk-max, default 32, per chunk), so a slow client holds neither an iterator nor a blocking thread. A new RpcIndexes::object_versions_iter, like the other *_iter reads, iterates one range of an object's versions, live objects and the deletion and wrap tombstones (ObjectOrTombstone); each chunk opens it for the versions it has left and takes up to chunk-max items. Each tombstone is listed as a removal (DELETED or WRAPPED).

Floor rule. The floor checkpoint F is the lowest checkpoint whose object state is intact: the floor GetObject's at_checkpoint reads already use (one below the higher of the lowest checkpoint with object data and the lowest with index history). Pruning removes the versions an object replaced, and the tombstones written, at or before the end of F, and keeps its live version at the end of F. Removals written after F are retained items. When anything below F is gone:

  • The object's oldest retained version is its version at the end of F. An object with no version there was created after F, or removed at or before F and then unwrapped, or created again under the same ID, after F; pruning drops a removed object's index rows. Its first item after F decides which. Items are placed through the rpc indexes alone: a live version by its transaction's sequence number (tx_seq_by_digest) and that transaction's checkpoint (tx_metadata_by_seq), a deletion by its removal transaction (see below). The indexes prune both in the same write that raises the floor, so an item they can no longer place was written at or before F and is skipped. A live version whose transaction created the object under a new ID keeps the whole history. object::new derives every ID it creates from the creating transaction's digest and a count (ObjectID::derive_id), so a new ID is one that matches some count the transaction could reach. An ID that matches no count costs all 32,768 counts the system-transaction limit allows, about 5.6 ms in a release build. So when the creating transaction's effects are kept, they settle the common cases first:

    • an object they don't list as created;
    • a derived object, whose claim adds a Field<derived_object::Claimed, ClaimedStatus> named after it to the parent in the same transaction.

    A dynamic field is ruled out by its type before any of that, and a new ID matches within the first few counts. Only effects that are pruned, or a derived object whose claim marker came from an earlier transaction, run the whole loop. Anything else makes that item the oldest retained one:

    • an unwrap;
    • a creation under an ID that can be used again: a dynamic field (removed and added again) or an object claimed through sui::derived_object (released and claimed again). Their IDs are hashes of a parent and a key, which never match a derived ID. A first creation looks the same as a re-creation, so every such object created after F is listed from its first version;
    • a first item that is a removal (an unwrap and deletion in one transaction, whose effects don't show whether the object was created wrapped).
  • If the listing's range starts below the oldest retained version, an ascending listing fails before listing anything, with start_version or an after cursor or neither.

  • In the same case, a descending listing lists every retained item in range, then fails instead of sending a QueryEnd, unless it reaches ITEM_LIMIT first.

  • Both fail with OUT_OF_RANGE / BELOW_EARLIEST_AVAILABLE, lowestAvailableCheckpoint = F and the new lowestAvailableVersion = the oldest retained version, so a client can retry with start_version set to it. A range starting at or above that version is served normally.

  • An object with no retained item at all, live or removal (deleted or wrapped at or before F, or never created; the node cannot tell which), fails with BELOW_EARLIEST_AVAILABLE and only lowestAvailableCheckpoint. That check runs before any cursor is compared with the tip. A node that has pruned no objects knows such an object never existed, and lists nothing, even for a listing whose transaction floor is set.

  • at_checkpoint below F fails as GetObject's does. A node running the objects compactor answers UNIMPLEMENTED / NOT_SERVED_HERE (cause no_object_history), one without rpc indexes UNIMPLEMENTED / NOT_SERVED_HERE (no_rpc_indexes), and one still backfilling index history after a restore UNAVAILABLE / NOT_READY (restore_backfill), through the checks at_checkpoint reads use.

Removal transactions. Object IDs can be reused: dynamic fields and derived objects can be deleted and created again under the same ID, at a higher version. Nothing changes a deleted object until a transaction creates it again, nor a wrapped one until a transaction unwraps or deletes it. That transaction writes the object's next item, unless it creates the object straight into a wrapper (a derived object claimed into a wrapper, with effects (NotExist, NotExist, Created)), which changes the object without writing an item of it. So the handler walks the object's transactions in the transaction bitmap's affected-object dimension back from the transaction of its next item, or, with no next item, from the end of the indexed transactions:

  • A next live version's transaction is placed through a new RpcIndexes::ledger_tx_seq_by_digest (the rpc-store's tx_seq_by_digest); a next tombstone through its own removal.
  • The first candidate whose effects list the object deleted or wrapped at that version is the removal. A candidate whose effects show otherwise is passed over.
  • The first candidate whose effects are pruned is the removal, unconfirmed: effects are pruned oldest checkpoint first, so no earlier candidate's are kept. The listing reports a removal's transaction only when it is confirmed; the floor rule places a deletion by either. A listing that reports transactions never meets an unconfirmed removal above its floor (see decision 22 below).
  • The lookup gives nothing when the next item's transaction, a candidate's bitmap entry or its index row is pruned. The indexes prune a bitmap entry no sooner than its row, so the walk never passes over a removal whose entry is gone without stopping at a pruned row first.

The next item is read from storage, not from the page, so the answer doesn't depend on paging or chunking. Every lookup in a listing is bounded by the highest indexed checkpoint as of planning, so it reads only what the indexes have committed.

Decisions.

  • Floor from the index plus a creation check. The oldest retained version is the object's version at the end of F in object_version_by_checkpoint. This ties the floor to the checkpoint the other at-checkpoint reads use. For objects with no row there, Nick chose to tell a creation from an unwrap by looking at the first version after F, over having the pruner keep each removed object's tombstone row forever. The creation check derives IDs from the transaction digest, so it still works when the creating effects are pruned. The cost is that objects whose IDs can be reused are treated as truncated.
  • Items placed through the rpc indexes, not checkpoint summaries. The rpc indexes prune their transaction rows in the same write that raises F, so "no row" means "at or before F". Checkpoint summaries and effects are pruned on their own schedule, which on nodes that keep every object version (num_epochs_to_retain = u64::MAX) runs above F.
  • Removal lookup by walking the bitmap, not a new index. A per-removal (object, version) → transaction index, which is how kv-rpc stores it (the digest in its tombstone), was rejected: a new history pipeline has no watermark on an existing store, so every pruned fullnode would wipe and rebuild its embedded rpc-store at upgrade. Changing the shared bitmap index to skip re-claims into a wrapper was rejected too: it would need a reindex.
  • Removal transactions are opt-in (decision 22). Finding a removal's transaction costs several reads per removal (the bitmap, the next item's transaction, the digest, the effects), so it runs only when removal_read_mask selects transaction; with the default version,kind nothing looks for it. A listing that asks for it has the higher of two floors: the object floor, and the checkpoint before the lowest one with removal transactions (GetServiceInfo's lowest_available_checkpoint, minus one). Both floors name the checkpoint at whose end the listing's history starts: the listing starts from each object's state at the end of it, and everything it lists after that was written in later checkpoints, which have their removal transactions. That is the same convention as the object floor (AtCheckpointLowerBound::pruned_floor, the checkpoint before its lowest available one), and lowestAvailableCheckpoint in the refusal names it. Taking the lowest checkpoint with removal transactions itself would refuse that checkpoint's removals for no reason. The two floors differ only on nodes that keep every object version and prune checkpoints. There the version index still holds an object's removal at or before the higher floor, whose transaction is pruned, so such an object has no version at that floor: its oldest retained version is its first item after the removal, every one of which was written after the floor, or, with none, it keeps nothing. Above the floor, a removal whose transaction cannot be found fails the listing with INTERNAL, after the floor is read again, so pruning that raced the read gets the floor error instead (but see the known gap below). A missing summary of the indexed tip checkpoint, which bounds the walk, is INTERNAL too.
  • Tombstones that pruning hasn't deleted yet are skipped. The rpc-store floor commits before the perpetual delete batch, and killswitch_tombstone_pruning keeps tombstones indefinitely, so the search for the oldest retained version can meet tombstones written at or before F. That search runs only for an object with no version at the end of F.
    • A WRAPPED tombstone it meets is skipped: a wrap after F would follow a live version written after F, which comes first.
    • A DELETED tombstone is placed by its removal transaction's checkpoint. A deletion the indexes can no longer place is at or before F and skipped. A retained deletion with pruned effects still sets the floor, through its unconfirmed removal. If that unconfirmed transaction is a re-claim into a wrapper after F while the deletion itself was at or before F, the listing starts at the stored deletion, and every item after it was written after F. A deletion at or before F followed by a re-creation of the ID after F is skipped, and the floor falls on the re-creation.
  • Pruning while a listing streams. The floor rises no later than the versions are deleted: the pruner commits the rpc-store floor before the perpetual batch that deletes them. So after each chunk the handler reads the floor again, and fails with the floor error if the oldest retained version moved above the lowest version the chunk read. If pruning removed every item of the object, the listing fails with BELOW_EARLIEST_AVAILABLE and only lowestAvailableCheckpoint, as for an object with nothing retained, rather than ending as if it had listed everything. Each floor read is repeated until the floor is the same before and after it, so the lookup never sees a half-applied prune. A descending listing's final floor error is built from the last chunk's floor read, so it names the current lowestAvailableCheckpoint.
  • Read through the object cache, from the perpetual objects table. The embedded rpc-store does not hold object bytes, so the read goes through AuthorityState's object cache like the other object reads, as a new ObjectCacheRead::object_versions_iter. The writeback cache holds no ranges, so that method reads the perpetual table alone and sees only committed versions. That is enough because the indexer receives a checkpoint only after its objects are committed. The executor broadcasts a checkpoint after committing it, and PerpetualStoreIngestionClient now refuses checkpoints above the highest executed one. That client could previously index a checkpoint whose data was readable only from the cache.
  • On RpcIndexes, not RpcStateReader. The listing needs the indexes for its checkpoint bound anyway. Backends without indexes (simulacrum, RestReadStore, the transactional test runner) answer UNIMPLEMENTED / NOT_SERVED_HERE (no_rpc_indexes) before any version read, and do not need an implementation.
  • sui-fork. Its object history starts at the fork checkpoint, which is its floor F. Its local rows are complete from an object's version at the fork checkpoint upward, so it lists from its stock reader.
  • Removals indexed below their tombstone. For a transaction with V1 effects, the version index records a deletion or wrap at lamport_version, which V1 effects recompute from the modified objects alone, below the tombstone the object store holds. The listing resolves an indexed version with no row to the tombstone just above it, as GetObject does, so such a removal is still listed when it is the object's last item.
  • Tests on real stores, not a stub. The floor and removal tests run on a fullnode pruned by the node's own pruning code. A test-only AuthorityStore::prune_objects_and_rpc_store_for_testing prunes the objects and the rpc indexes through a checkpoint as the background pruner does, keeping tombstones or not, so tests control where F lands. Two fail points, prune-objects-before-perpetual-commit and list-object-versions-after-plan, compile to nothing outside the simulator; they pause a prune between its two commits and a listing between its plan and its first chunk.
  • ChunkedScan gains a terminal type parameter, because a version listing has no ledger watermark to carry. The existing List APIs keep ScanTerminal as the default.

Parity with kv-rpc. Both backends parse requests, plan ranges, pick QueryEnd reasons, mint cursors (ObjectVersionCursor) and emit frames with the same sui-rpc-api code. They give the same answer for every object whose history the fullnode retains. A cursor from either backend decodes on the other. The shared plan handles a cursor above the object's version at the indexed tip as the other List* RPCs do: a descending listing fails with ABOVE_LATEST_AVAILABLE, and an ascending one is served from this server's view and, unless end_version or before ends it first, ends with LEDGER_TIP. On the fullnode the tip is its highest indexed checkpoint. Only the fullnode applies the floor rules, which follow #43's contract: history is kept from each object's state at the end of lowestAvailableCheckpoint, and the server may report a higher oldest retained version than an object strictly needs.

Impact and rollout. Fullnodes start answering ListObjectVersions. New config ledger-history.list-object-versions (timeout-ms, default-limit-items 50, max-limit-items 1000, chunk-max 32). Its chunks are timed in the shared list_chunk_seconds histogram, with method="list_object_versions". No protocol change.

Test coverage. Fullnode simtests read two versions per chunk, so every listing crosses chunk boundaries. They cover:

  • an object through create, mutate, wrap, unwrap, mutate and delete: its live versions and its WRAPPED and DELETED removals with their transactions, listed in both orders and paged by cursor with every removal's transaction intact on every page size. Also the wrapper, deleted by the unwrap;
  • at_checkpoint at every checkpoint of the lifecycle;
  • version ranges, the CURSOR_BOUND, CHECKPOINT_BOUND and LEDGER_TIP ends, an ascending after past the tip ending with LEDGER_TIP, and a descending before past the tip failing without at_checkpoint and served with it;
  • the clock object's full history checked against its version at every checkpoint, ordered and free of duplicates;
  • after a real zero-retention prune:
    • the ascending error with and without a lower bound, and with end_version at the oldest retained version;
    • a retry from the oldest retained version;
    • descending listings that list the retained versions then fail, including a range wholly below the floor;
    • start_version exactly at the oldest retained version, and ITEM_LIMIT at the floor;
    • at_checkpoint at and below the floor;
    • refusal of an object with no retained versions, with and without a cursor past the tip;
  • an object created after a real prune, then wrapped, unwrapped and deleted, listed in full with its removals' transactions in both orders, through the node's own effects, rpc index and bitmap reads;
  • a derived object claimed after a real prune, released and claimed again under the same ID: both incarnations and the release's transaction are listed, and listings below the first claim fail, because a derived object's ID can be reused;
  • a derived object released at or before the floor, its tombstone kept, and claimed again after it: listed from the re-claim, passing over the leftover tombstone;
  • the same, pruned by the fullnode's own background pruner (which also retracts the rpc indexes). Simulator only, because the background pruner runs every few seconds only under msim;
  • an object wrapped at or before the floor and unwrapped after it, with the wrap's tombstone pruned and kept: listed from the unwrap;
  • a derived object claimed, released, claimed again straight into a wrapper and released from it: both removals listed with their own transactions, past the re-claim that wrote no item;
  • the same with the first release at or before the floor and its tombstone kept: listed from the last release, with its transaction;
  • the same on a node that also pruned checkpoints above the object floor, so the last release's effects are gone. Without transactions, it is listed from the last release. With them, the floor is the checkpoint floor, at or after the last release, and the object keeps nothing; once it is claimed and released again after that floor, it is listed from the new claim, with the new release's transaction;
  • an object's removals listed with the default removal mask, without their transactions;
  • while a prune is paused between its rpc-store commit and its perpetual batch (simulator only): an unwrapped object and a derived object claimed and released twice before the floor and claimed after it are listed from their first item after the floor, passing over the leftover versions and tombstones;
  • pruning between a listing's plan and its first chunk (simulator only): one prune removes every item of the object and fails the listing with BELOW_EARLIEST_AVAILABLE; another raises the oldest retained version above where the listing starts and fails it with the new version.

Unit tests cover the listing floor with and without removal transactions, the creation check's ID derivation and claim marker, the resolution of a removal indexed below its tombstone, the sui-rpc-store range read with tombstones, the ingestion client's executed-tip gate, and the bitmap rule the removal walk relies on: a transaction that only references a deleted shared object adds no AffectedObject entry for it.

Not covered by a test:

  • a wrap and an unwrap in the same checkpoint;
  • pruning between two chunks of one listing (the tests pause only before the first chunk);
  • a re-claim into a wrapper followed by an unwrap, rather than a release;
  • the final floor error of a descending listing refreshed from its last chunk;
  • removals indexed below their tombstone on a real store (V1 effects can't be produced there; only the unit test covers it);
  • a fork listing;
  • a removal above the floor whose transaction cannot be found, which is INTERNAL: no real store reaches it outside the race below. A mutation that drops the transaction floor reaches it, and list_object_versions_deleted_with_pruned_effects fails on that mutation;
  • an object that never existed, listed with removal transactions on a node that has pruned checkpoints but no objects.

Known gap: AuthorityStorePruner::prune_checkpoints deletes the effects (perpetual_batch) before it raises the checkpoint floor (checkpoints_batch, HighestPruned). A listing that asks for transactions and reads a removal between those two writes finds its effects gone while the floor recheck still sees the old floor, and fails with INTERNAL instead of the floor error. Writing the checkpoint batch first would close the window, but a crash between the two writes would then leave that checkpoint's effects unpruned for good. The pruner's write order is a follow-up.

Test plan

New simtests:

  • list_object_versions_lifecycle
  • list_object_versions_below_oldest_retained_version
  • list_object_versions_created_after_the_floor
  • list_object_versions_reused_id
  • list_object_versions_claimed_again_after_the_floor
  • list_object_versions_claimed_again_after_background_pruning (simulator only)
  • list_object_versions_unwrapped_after_the_floor
  • list_object_versions_reclaimed_into_a_wrapper
  • list_object_versions_reclaimed_into_a_wrapper_after_the_floor
  • list_object_versions_deleted_with_pruned_effects
  • list_object_versions_leftovers_while_pruning (simulator only)
  • list_object_versions_pruned_while_listing (simulator only)
  • list_object_versions_floor_rises_while_listing (simulator only)

New unit tests:

  • created_under_new_id_tells_reusable_ids_apart
  • removal_indexed_below_its_tombstone_resolves_to_it
  • object_versions_iter_lists_tombstones_within_bounds
  • checkpoint_above_highest_executed_is_not_found
  • transaction_visitor_skips_consensus_objects_whose_stream_ended
  • listing_floor_is_the_checkpoint_before_transaction_data_when_higher

Release notes

  • Protocol:
  • Nodes (Validators and Full nodes):
  • gRPC: Full nodes serve LedgerService.ListObjectVersions for objects within their retention window, including the versions at which an object was deleted or wrapped, and, when removal_read_mask selects transaction, the transaction that removed it. A listing that would start or end inside pruned history fails with OUT_OF_RANGE / BELOW_EARLIEST_AVAILABLE, naming the oldest retained version in lowestAvailableVersion, or, for an object the node retains nothing of, only lowestAvailableCheckpoint. On a node that prunes checkpoints sooner than objects, a listing that asks for removal transactions has the higher of the two floors.
  • JSON-RPC:
  • GraphQL:
  • CLI:
  • Rust SDK:
  • Indexing Framework:

@nickvikeras
nickvikeras force-pushed the nickv/list-object-versions branch from c90a27d to 0619a00 Compare November 11, 2026 17:15
@nickvikeras
nickvikeras force-pushed the nickv/list-object-versions-fullnode branch from 21ae310 to de1a529 Compare November 11, 2026 17:30
nickvikeras added a commit that referenced this pull request Nov 11, 2026
# Conflicts:
#	Cargo.lock
#	Cargo.toml
#	crates/sui-fork/src/tests/store_execution.rs
#	crates/sui-indexer-alt-e2e-tests/tests/kv_rpc_tests.rs
#	crates/sui-kvstore/src/bigtable/client/mod.rs
#	crates/sui-rpc-api/src/reader.rs
#	crates/sui-types/src/storage/read_store.rs
@nickvikeras
nickvikeras force-pushed the nickv/list-object-versions branch from 0619a00 to 7cd7deb Compare November 12, 2026 18:21
@nickvikeras
nickvikeras force-pushed the nickv/list-object-versions-fullnode branch 2 times, most recently from 155145a to 78d06ac Compare November 12, 2026 18:53
@nickvikeras
nickvikeras force-pushed the nickv/list-object-versions branch from 7cd7deb to 8fc0f03 Compare November 12, 2026 18:59
@nickvikeras
nickvikeras force-pushed the nickv/list-object-versions-fullnode branch from 78d06ac to 209f2d2 Compare November 12, 2026 19:06
nickvikeras added a commit that referenced this pull request Nov 12, 2026
# Conflicts:
#	Cargo.lock
#	Cargo.toml
#	crates/sui-fork/src/tests/store_execution.rs
#	crates/sui-indexer-alt-e2e-tests/tests/kv_rpc_tests.rs
#	crates/sui-kvstore/src/bigtable/client/mod.rs
#	crates/sui-kvstore/src/bigtable/mock_server.rs
#	crates/sui-rpc-api/src/reader.rs
#	crates/sui-types/src/storage/read_store.rs
nickvikeras added a commit that referenced this pull request Nov 12, 2026
# Conflicts:
#	Cargo.lock
#	Cargo.toml
#	crates/sui-fork/src/tests/store_execution.rs
#	crates/sui-indexer-alt-e2e-tests/tests/kv_rpc_tests.rs
#	crates/sui-kvstore/src/bigtable/client/mod.rs
#	crates/sui-kvstore/src/bigtable/mock_server.rs
#	crates/sui-rpc-api/src/reader.rs
#	crates/sui-types/src/storage/read_store.rs
@nickvikeras
nickvikeras force-pushed the nickv/list-object-versions branch from 8fc0f03 to 0a1a459 Compare November 12, 2026 20:20
@nickvikeras
nickvikeras force-pushed the nickv/list-object-versions-fullnode branch from 209f2d2 to d657237 Compare November 12, 2026 20:33
nickvikeras added a commit that referenced this pull request Nov 12, 2026
# Conflicts:
#	Cargo.lock
#	Cargo.toml
#	crates/sui-fork/src/tests/store_execution.rs
#	crates/sui-indexer-alt-e2e-tests/tests/kv_rpc_tests.rs
#	crates/sui-kvstore/src/bigtable/client/mod.rs
#	crates/sui-kvstore/src/bigtable/mock_server.rs
#	crates/sui-rpc-api/src/reader.rs
#	crates/sui-types/src/storage/read_store.rs
@nickvikeras
nickvikeras force-pushed the nickv/list-object-versions-fullnode branch from d657237 to ed7b7fe Compare November 12, 2026 21:09
nickvikeras added a commit that referenced this pull request Nov 12, 2026
# Conflicts:
#	Cargo.lock
#	Cargo.toml
#	crates/sui-fork/src/tests/store_execution.rs
#	crates/sui-indexer-alt-e2e-tests/tests/kv_rpc_tests.rs
#	crates/sui-kvstore/src/bigtable/client/mod.rs
#	crates/sui-kvstore/src/bigtable/mock_server.rs
#	crates/sui-rpc-api/src/reader.rs
#	crates/sui-types/src/storage/read_store.rs
@nickvikeras
nickvikeras force-pushed the nickv/list-object-versions branch from 0a1a459 to 6a71e2f Compare November 12, 2026 21:17
@nickvikeras
nickvikeras force-pushed the nickv/list-object-versions-fullnode branch from ed7b7fe to 2e8b0d6 Compare November 12, 2026 21:34
nickvikeras added a commit that referenced this pull request Nov 12, 2026
# Conflicts:
#	Cargo.lock
#	Cargo.toml
#	crates/sui-fork/src/tests/store_execution.rs
#	crates/sui-indexer-alt-e2e-tests/tests/kv_rpc_tests.rs
#	crates/sui-kvstore/src/bigtable/client/mod.rs
#	crates/sui-kvstore/src/bigtable/mock_server.rs
#	crates/sui-rpc-api/src/reader.rs
#	crates/sui-types/src/storage/read_store.rs
@nickvikeras
nickvikeras force-pushed the nickv/list-object-versions-fullnode branch from 2e8b0d6 to 2c1824a Compare November 12, 2026 22:11
nickvikeras added a commit that referenced this pull request Nov 12, 2026
# Conflicts:
#	Cargo.lock
#	Cargo.toml
#	crates/sui-fork/src/tests/store_execution.rs
#	crates/sui-indexer-alt-e2e-tests/tests/kv_rpc_tests.rs
#	crates/sui-kvstore/src/bigtable/client/mod.rs
#	crates/sui-kvstore/src/bigtable/mock_server.rs
#	crates/sui-rpc-api/src/reader.rs
#	crates/sui-types/src/storage/read_store.rs
@nickvikeras
nickvikeras force-pushed the nickv/list-object-versions branch from 6a71e2f to 82c8788 Compare November 12, 2026 22:30
@nickvikeras
nickvikeras force-pushed the nickv/list-object-versions-fullnode branch from 2c1824a to cabbda9 Compare November 12, 2026 22:46
nickvikeras added a commit that referenced this pull request Nov 12, 2026
# Conflicts:
#	Cargo.lock
#	Cargo.toml
#	crates/sui-fork/src/tests/store_execution.rs
#	crates/sui-indexer-alt-e2e-tests/tests/kv_rpc_tests.rs
#	crates/sui-kvstore/src/bigtable/client/mod.rs
#	crates/sui-kvstore/src/bigtable/mock_server.rs
#	crates/sui-rpc-api/src/reader.rs
#	crates/sui-types/src/storage/read_store.rs
@nickvikeras
nickvikeras force-pushed the nickv/list-object-versions branch from 82c8788 to a8fa55b Compare November 12, 2026 23:02
@nickvikeras
nickvikeras force-pushed the nickv/list-object-versions-fullnode branch from cabbda9 to 6d49ee5 Compare November 12, 2026 23:12
nickvikeras added a commit that referenced this pull request Nov 12, 2026
# Conflicts:
#	Cargo.lock
#	Cargo.toml
#	crates/sui-fork/src/tests/store_execution.rs
#	crates/sui-indexer-alt-e2e-tests/tests/kv_rpc_tests.rs
#	crates/sui-kvstore/src/bigtable/client/mod.rs
#	crates/sui-kvstore/src/bigtable/mock_server.rs
#	crates/sui-rpc-api/src/reader.rs
#	crates/sui-types/src/storage/read_store.rs
@nickvikeras
nickvikeras force-pushed the nickv/list-object-versions branch from a8fa55b to be6c21d Compare November 12, 2026 23:54
@nickvikeras
nickvikeras force-pushed the nickv/list-object-versions-fullnode branch from 6d49ee5 to 734593f Compare November 13, 2026 00:18
@nickvikeras
nickvikeras force-pushed the nickv/list-object-versions branch 2 times, most recently from 3fdf275 to 6dc4790 Compare November 13, 2026 01:02
@nickvikeras
nickvikeras force-pushed the nickv/list-object-versions-fullnode branch from 734593f to b69b163 Compare November 13, 2026 01:10
nickvikeras added a commit that referenced this pull request Nov 13, 2026
# Conflicts:
#	Cargo.lock
#	Cargo.toml
#	crates/sui-fork/src/tests/store_execution.rs
#	crates/sui-indexer-alt-e2e-tests/tests/kv_rpc_tests.rs
#	crates/sui-kvstore/src/bigtable/client/mod.rs
#	crates/sui-kvstore/src/bigtable/mock_server.rs
#	crates/sui-kvstore/src/tables/objects.rs
#	crates/sui-rpc-api/src/reader.rs
#	crates/sui-types/src/storage/read_store.rs
@nickvikeras
nickvikeras force-pushed the nickv/list-object-versions-fullnode branch from b69b163 to cbbf11a Compare November 13, 2026 04:15
@nickvikeras
nickvikeras force-pushed the nickv/list-object-versions branch 2 times, most recently from b24c252 to 9c078b4 Compare November 13, 2026 04:53
@nickvikeras
nickvikeras force-pushed the nickv/list-object-versions-fullnode branch from cbbf11a to 698c3ef Compare November 13, 2026 04:53
@nickvikeras
nickvikeras force-pushed the nickv/list-object-versions branch from 9c078b4 to 9a6e75f Compare November 13, 2026 05:02
@nickvikeras
nickvikeras force-pushed the nickv/list-object-versions-fullnode branch 2 times, most recently from 44cdb68 to ba3452c Compare November 13, 2026 11:11
@nickvikeras
nickvikeras force-pushed the nickv/list-object-versions branch from 9a6e75f to 9721a19 Compare November 13, 2026 11:11
@nickvikeras
nickvikeras force-pushed the nickv/list-object-versions-fullnode branch from ba3452c to f34e4e1 Compare November 14, 2026 01:04
@nickvikeras
nickvikeras force-pushed the nickv/list-object-versions branch 2 times, most recently from 1912f75 to ecb0307 Compare November 14, 2026 06:25
@nickvikeras
nickvikeras force-pushed the nickv/list-object-versions-fullnode branch from f34e4e1 to 1b966b0 Compare November 14, 2026 06:25
@nickvikeras
nickvikeras force-pushed the nickv/list-object-versions branch from ecb0307 to d4a6aaf Compare November 14, 2026 18:50
@nickvikeras
nickvikeras force-pushed the nickv/list-object-versions-fullnode branch 2 times, most recently from b530dec to b0e9aa5 Compare November 15, 2026 02:09
@nickvikeras
nickvikeras force-pushed the nickv/list-object-versions branch from d4a6aaf to 509c3be Compare November 15, 2026 02:09
…window

Fullnodes keep every object version within their retention window, so
they serve ListObjectVersions for the objects whose history they retain
instead of answering UNIMPLEMENTED. The handler reuses #28632's
ObjectVersionsQuery, plan and frames, and reads items in chunks of
blocking reads of the perpetual objects table, through a new
ObjectCacheRead::object_versions_iter.

Below the retention floor F, the checkpoint at whose end object state is
intact (as for at_checkpoint reads), an object's oldest retained version
is its version at the end of F, or, for an object with no version there,
its first item after F unless that item is a creation under a new ID. A
range starting below it fails with OUT_OF_RANGE / BELOW_EARLIEST_AVAILABLE,
with lowestAvailableCheckpoint and a new lowestAvailableVersion; an object
with nothing retained fails with lowestAvailableCheckpoint only. Items are
placed through the rpc indexes, which prune in the same write that raises
F, and the floor is read again after each chunk, so a prune during a
listing fails it instead of cutting it short.

A removal's transaction is found by walking the object's transactions in
the bitmap's affected-object dimension back from its next item, and only
when removal_read_mask selects transaction. Such a listing's floor is the
higher of F and the checkpoint before the lowest one with removal
transactions (lowest_available_checkpoint - 1), and a removal above it
whose transaction cannot be found fails the listing with INTERNAL.

Adds RpcIndexes::ledger_tx_seq_by_digest, and for the tests
AuthorityStore::prune_objects_and_rpc_store_for_testing and two fail
points, compiled only under msim or the fail_points feature.
@nickvikeras
nickvikeras force-pushed the nickv/list-object-versions branch from 509c3be to 4806862 Compare November 15, 2026 19:59
@nickvikeras
nickvikeras force-pushed the nickv/list-object-versions-fullnode branch from b0e9aa5 to 0ee2270 Compare November 15, 2026 19:59

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant