Repository navigation
Conversation
There was a problem hiding this comment.
❌ Gate recommendation: request changes.
The publication condition needs to remain valid across history cleanup. Fence the durable head/high-watermark atomically, or coordinate cleanup with the full strict publication window, before advertising these handlers as strict expected-head capable.
| } | ||
|
|
||
| fn supports_expected_head_commit(&self) -> bool { | ||
| true |
There was a problem hiding this comment.
Checking that expected + 1 is unoccupied does not preserve the expected-head guarantee when cleanup runs concurrently. After the strict preflight reads version 1, two compatible commits can publish versions 2 and 3, and ordinary cleanup can remove version 2. RenameCommitHandler then lets the strict writer recreate version 2 and return success while the actual head remains 3. Snapshot-dependent publishers can consequently receive success for output that never became the current dataset head, violating #8699's no-intervening-commit contract. Conditional PUT and lock-backed commits rely on the same removable manifest slot.
The publication fence must survive removal of that slot. A durable head/high-watermark condition or coordination with cleanup is needed; another unlocked head check leaves the race open.
Executed regression test
Add this to the existing rust/lance/src/io/commit.rs test module, using this PR's strict_dataset and empty_append helpers. Run cargo test -p lance --lib test_strict_expected_head. The six added PR tests pass; this test fails with strict commit returned success at Ok(2) after head advanced to 3. Cleanup uses its default verification settings; the dataset is never deleted or recreated.
#[derive(Debug)]
struct GateCleanupRaceHandler {
advanced: Mutex<Option<Dataset>>,
}
#[async_trait::async_trait]
impl CommitHandler for GateCleanupRaceHandler {
fn supports_expected_head_commit(&self) -> bool {
true
}
async fn commit(
&self,
manifest: &mut Manifest,
indices: Option<Vec<IndexMetadata>>,
base_path: &Path,
object_store: &ObjectStore,
manifest_writer: ManifestWriter,
naming_scheme: ManifestNamingScheme,
transaction: Option<lance_table::format::Transaction>,
) -> std::result::Result<ManifestLocation, CommitError> {
let mut advanced = self.advanced.lock().unwrap().take().unwrap();
for _ in 0..2 {
let read_version = advanced.version_id();
advanced = CommitBuilder::new(Arc::new(advanced))
.execute(empty_append(read_version))
.await?;
}
let policy = CleanupPolicyBuilder::default()
.versions(vec![manifest.version])?
.build();
crate::dataset::cleanup::cleanup_old_versions(&advanced, policy).await?;
assert_eq!(advanced.latest_version_id().await?, manifest.version + 1);
assert!(object_store.inner.head(&naming_scheme.manifest_path(base_path, manifest.version)).await.is_err());
RenameCommitHandler
.commit(manifest, indices, base_path, object_store, manifest_writer, naming_scheme, transaction)
.await
}
}
#[tokio::test]
async fn test_strict_expected_head_gate_cleanup_race() {
let tmp = TempStrDir::default();
let dataset = strict_dataset(tmp.as_str()).await;
let expected = dataset.version_id();
let handler = Arc::new(GateCleanupRaceHandler {
advanced: Mutex::new(Some(dataset.clone())),
});
let result = CommitBuilder::new(Arc::new(dataset))
.with_commit_handler(handler)
.with_expected_latest_version(expected)
.execute(empty_append(expected))
.await;
let actual_head = Dataset::open(tmp.as_str()).await.unwrap().version_id();
assert_eq!(actual_head, expected + 2);
assert!(result.is_err(), "strict commit returned success at {:?} after head advanced to {actual_head}", result.as_ref().map(|ds| ds.version_id()));
}There was a problem hiding this comment.
The added post-publication check catches the original successful-return reproducer, but the same cleanup race still reports success after an ambiguous write error. Err(CommitError::OtherError(_)) → CommitOutcome::Ours bypasses strict_head_is_published, so a lost response can acknowledge version 2 after the head advanced to 3 and cleanup freed version 2. Built-in rename and conditional-PUT handlers suppress such errors after readback. This still violates #8699's strict publication contract. The publication fence must apply independently of the handler's response.
Reproducer
This wraps the PR's GateCleanupRaceHandler to model a lost response after its manifest write. Add it to the same test module and run cargo test -p lance --lib test_strict_expected_head_gate_cleanup_ambiguous.
#[derive(Debug)]
struct GateCleanupAmbiguousHandler {
inner: GateCleanupRaceHandler,
}
#[async_trait::async_trait]
impl CommitHandler for GateCleanupAmbiguousHandler {
fn supports_expected_head_commit(&self) -> bool { true }
fn propagate_commit_error_after_success(&self) -> bool { false }
async fn commit(
&self,
manifest: &mut Manifest,
indices: Option<Vec<IndexMetadata>>,
base_path: &Path,
object_store: &ObjectStore,
manifest_writer: ManifestWriter,
naming_scheme: ManifestNamingScheme,
transaction: Option<lance_table::format::Transaction>,
) -> std::result::Result<ManifestLocation, CommitError> {
self.inner.commit(manifest, indices, base_path, object_store,
manifest_writer, naming_scheme, transaction).await?;
Err(CommitError::OtherError(Error::io("injected lost response")))
}
}
#[tokio::test]
async fn test_strict_expected_head_gate_cleanup_ambiguous() {
let tmp = TempStrDir::default();
let dataset = strict_dataset(tmp.as_str()).await;
let expected = dataset.version_id();
let handler = Arc::new(GateCleanupAmbiguousHandler {
inner: GateCleanupRaceHandler {
advanced: Mutex::new(Some(dataset.clone())),
},
});
let result = CommitBuilder::new(Arc::new(dataset))
.with_commit_handler(handler)
.with_expected_latest_version(expected)
.execute(empty_append(expected)).await;
let actual_head = Dataset::open(tmp.as_str()).await.unwrap().version_id();
assert_eq!(actual_head, expected + 2);
assert!(matches!(result, Err(Error::RetryableCommitConflict { .. })),
"strict result {:?} while actual head is {actual_head}",
result.as_ref().map(|ds| ds.version_id()));
}Executed on 2de2bb1: fails with strict result Ok(2) while actual head is 3. The seven PR strict-mode tests pass.
A normal non-overwrite commit can rebase a transaction before its first attempt, accepting a concurrent compatible change that the caller never validated. Disabling retries does not close this window because the initial rebase happens before the retry loop. Expose the existing conditional commit path through CommitBuilder::with_expected_latest_version. The transaction read version must equal the expected version; if the current head differs, the first attempt returns a retryable conflict. Strict commits use one attempt, skip the TransactionRebase path, and therefore cannot replace the original expected-head witness. New datasets, detached commits, and mismatched read versions are rejected before publication. Strict mode also requires a commit handler that can reject a concurrent write for the same manifest version. Add CommitHandler::supports_expected_head_commit and enable it for conditional-put, rename, commit-lock, and external-manifest handlers. UnsafeCommitHandler and other handlers that do not opt in return an unsupported error instead of claiming a guarantee they cannot provide. Expose the mode through the Python LanceDataset.commit API and the Java CommitBuilder API, including tests and documentation. Add tests for unchanged and changed heads, an injected concurrent commit before publication, retry configuration being ignored, rename handlers, unsafe handler rejection, and mismatched read versions.
There was a problem hiding this comment.
❌ Gate recommendation: request changes.
Two correctness issues remain in the strict publication contract. Enforce the expected-head condition at publication with a fence that survives cleanup, and preserve successfully published manifests and transactions when a subsequent writer advances the head. A later head read cannot safely decide whether this attempt lost.
For the cleanup-exclusion approach, land #10504 first, then rebase and reuse its shared history guard across the full strict publication window.
| match strict_head_is_published(&dataset, target_version).await { | ||
| Ok(true) => {} | ||
| Ok(false) => { | ||
| rollback_unpublished_manifest( |
There was a problem hiding this comment.
A successful strict commit is deleted if another writer publishes its successor before this check. The writer can correctly publish version 2 against expected head 1, and a normal writer can then publish version 3 incorporating its changes. This branch deletes version 2 and its transaction and returns a retryable conflict even though the change remains applied in version 3. That loses readable version history and invites retries of an already-applied operation. The matching CommitConflict/Ours branch has the same problem.
Keep successfully published artifacts intact and decide the expected-head condition atomically when publishing; observing a successor afterward does not establish that this attempt lost.
Reproducer
Add this to the existing rust/lance/src/io/commit.rs test module, using the PR's strict_dataset and empty_append helpers. Run cargo test -p lance --lib test_strict_expected_head_gate_successor_keeps_publication.
#[derive(Debug)]
struct GateAdvanceAfterPublishHandler {
uri: String,
}
#[async_trait::async_trait]
impl CommitHandler for GateAdvanceAfterPublishHandler {
fn supports_expected_head_commit(&self) -> bool { true }
async fn commit(
&self,
manifest: &mut Manifest,
indices: Option<Vec<IndexMetadata>>,
base_path: &Path,
object_store: &ObjectStore,
manifest_writer: ManifestWriter,
naming_scheme: ManifestNamingScheme,
transaction: Option<lance_table::format::Transaction>,
) -> std::result::Result<ManifestLocation, CommitError> {
let location = RenameCommitHandler.commit(manifest, indices, base_path,
object_store, manifest_writer, naming_scheme, transaction).await?;
let published = Dataset::open(&self.uri).await?;
assert_eq!(published.version_id(), manifest.version);
CommitBuilder::new(Arc::new(published))
.execute(empty_append(manifest.version)).await?;
Ok(location)
}
}
#[tokio::test]
async fn test_strict_expected_head_gate_successor_keeps_publication() {
let tmp = TempStrDir::default();
let dataset = strict_dataset(tmp.as_str()).await;
let expected = dataset.version_id();
let transaction = Transaction::new(expected, Operation::UpdateConfig {
config_updates: None,
table_metadata_updates: Some(lance_table::transaction::UpdateMap {
update_entries: vec![("strict-output", "published").into()],
replace: false,
}),
schema_metadata_updates: None,
field_metadata_updates: HashMap::new(),
}, None);
let result = CommitBuilder::new(Arc::new(dataset))
.with_commit_handler(Arc::new(GateAdvanceAfterPublishHandler {
uri: tmp.as_str().to_owned(),
}))
.with_expected_latest_version(expected)
.execute(transaction).await;
let latest = Dataset::open(tmp.as_str()).await.unwrap();
assert_eq!(latest.version_id(), expected + 2);
assert_eq!(latest.manifest.table_metadata.get("strict-output").map(String::as_str),
Some("published"));
let historical = latest.checkout_version(expected + 1).await;
assert!(result.is_ok() && historical.is_ok(),
"valid publication returned {:?}; version {} readable: {} (error: {:?}); its output remains in version {}",
result.as_ref().map(|ds| ds.version_id()), expected + 1, historical.is_ok(),
historical.as_ref().err(), latest.version_id());
}Executed on 2de2bb1: the assertion fails with RetryableCommitConflict, version 2 readable: false (DatasetNotFound), and its output remains in version 3. No cleanup or dataset deletion is performed in this reproducer.
Summary
Add a strict expected-head commit mode for callers that validate a complete
snapshot-dependent basis before publishing output. Fixes #8699.
A normal non-overwrite commit may rebase a transaction before its first attempt.
That can accept a concurrent compatible change which landed after the caller's
validation.
with_max_retries(0)does not help because the rebase occurs beforethe retry loop.
What changed
CommitBuilder::with_expected_latest_version(expected).read_versionmust equalexpected.expected.Error::RetryableCommitConflict.with_max_retries.TransactionRebaseand no compatibility-based acceptance.with
InvalidInput.CommitHandler::supports_expected_head_commit.ConditionalPutCommitHandler: trueRenameCommitHandler: trueCommitLock: trueExternalManifestCommitHandler: trueUnsafeCommitHandler: falseNotSupportedfor handlers that cannot reject aconcurrent write for the same manifest version.
execute_batchtransactions against the expected version.expected_latest_versionin PythonLanceDataset.commitandexpectedLatestVersionin JavaCommitBuilder.Semantics
The target version is
expected + 1. A safe commit handler's conditional writecloses the race between the initial latest-version check and manifest
publication. If another writer occupies
expected + 1, the handler reports aconflict and strict mode does not retry.
After a strict conflict, the caller must reload and revalidate the dataset
before submitting a new transaction with a new expected version.
The current head witness is the version number. A version plus manifest identity
witness remains a possible future extension for ABA cases where a dataset is
deleted and recreated at the same version.
No protobuf or file-format change is required.
Tests
Added/updated:
test_strict_expected_head_commits_when_unchangedtest_strict_expected_head_rejects_changed_headtest_strict_expected_head_does_not_retry_after_concurrent_committest_strict_expected_head_requires_a_conflict_safe_handlertest_strict_expected_head_requires_matching_read_versiontest_strict_expected_head_supports_rename_handlertest_commit_with_expected_latest_version(Python)CommitBuilderExpectedHeadTest(Java)