Skip to content

feat: expose a public toggle for disable_transaction_file - #11232

Draft
vaibhavdabas16 wants to merge 1 commit into
lance-format:mainfrom
vaibhavdabas16:feat/7486-disable-transaction-file
Draft

vaibhavdabas16 wants to merge 1 commit into
lance-format:mainfrom
vaibhavdabas16:feat/7486-disable-transaction-file

Conversation

@vaibhavdabas16

Copy link
Copy Markdown

Closes #7486

Summary

disable_transaction_file was plumbed end to end but could only be turned on from tests. This adds a public switch:

  • WriteParams::disable_transaction_file (Rust), forwarded by InsertBuilder
  • CommitBuilder::with_disable_transaction_file(bool) (Rust)
  • disable_transaction_file= on lance.write_dataset (Python), as a thin wrapper over WriteParams

When set, the commit stores its transaction only in the manifest's inline transaction_section, skips the _transactions/ file, and sets FLAG_DISABLE_TRANSACTION_FILE. The setting is per commit, as the flag is not inherited: a later commit that leaves it unset writes the file again and drops the flag. The default stays false; turning it on by default is #7487.

Java bindings are not included and can follow in a separate PR with the same parameter name.

Tests

  • Rust: test_disable_transaction_file_applies_to_its_commit_only checks that no transaction file is written, the flag is set, a freshly opened dataset reads the inline transaction, and a following append writes the file and clears the flag.
  • Python: test_disable_transaction_file covers the same behavior through write_dataset.

Local verification

  • cargo fmt --all, ruff, ruff-format, and typos pass.
  • Not yet run locally: the new Rust and Python tests and cargo clippy. The local lance test build ran out of memory on my machine, so this is a draft until CI passes.

Add `WriteParams::disable_transaction_file` and
`CommitBuilder::with_disable_transaction_file` so a commit can store its
transaction only inline in the manifest and skip the redundant file under
`_transactions/`. The flag applies per commit: later commits that leave it
unset write the transaction file again and drop the writer feature flag.

Expose the same option as `disable_transaction_file` on Python's
`write_dataset`.

Closes lance-format#7486
@github-actions github-actions Bot added A-python Python bindings enhancement New feature or request labels Nov 18, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-python Python bindings enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Expose a public toggle for disable_transaction_file (skip redundant external transaction file)

1 participant