Repository navigation
Conversation
…tack 1.x
The existing compat suite showed protect-dynamodb 6.0.1 items read and
query under Stack 1.x, but its acceptance run re-encrypted them as a
backfill step, so it never showed the backfill could be skipped. A new
block freezes a protect-dynamodb corpus (single and bulk writes), moves
the app to Stack 1.x alone, decrypts every item, runs every equality
query, writes new items beside the old ones, and then asserts the stored
items are byte-identical to the snapshot.
The stash-dynamodb skill told customers to re-encrypt every item to move
a table to v3. That is not required, so the skill now says re-encryption
is optional and lists the three differences to plan for: boolean
equality lookups, downcase filters, and legacy dataType('json')
attributes.
The legacy client builder and the result unwrapper move to a shared
helper so the cross-store suite can reuse them.
An encrypted scalar is the same two strings in both stores: DynamoDB's <attr>__source and <attr>__hmac are the c and hm fields of a Postgres EQL v3 value. The rest of the Postgres value is the version and the table and column names, none of it secret, so moving a value between stores is a copy with no key involved. The new suite pins that in both directions with real protect-dynamodb 6.0.1 and protect 10.2.1 payloads as well as Stack 1.x ones: the EQL v3 column types accept the copied values, the ciphertext arrives byte-identical, Stack decrypts it, SQL equality queries and DynamoDB search hashes still match, and protect-dynamodb 6.0.1 reads items copied from Postgres. It needs CS_* credentials and DATABASE_URL, so it runs from its own config as test:protect-crossover, and the e2e-tests job now starts a Postgres through integration-db for it.
The boolean-equality test accepted any encryptQuery failure, so a service
outage passed it. It now asserts the "no indexes configured" refusal
Stack gives a storage-only column, after a control query on the same
client succeeds. The snapshot test claimed more than it could show:
nothing in its block writes to the table, so it is renamed and its
comments say it pins only that reads and queries left the stored items
unchanged.
The cross-store suite sorted results with ORDER BY, which follows the
database collation, so it now sorts in JavaScript. The protect-dynamodb
decrypt check uses toEqual like every other decrypt check, and the
raw-payload block goes through the same two copy functions as the rest
of the file instead of a third, slightly different one.
In CI the cross-store suite was skipped whenever the protect-dynamodb
step failed. Both new steps now run on !cancelled().
The stash-dynamodb skill said Stack reads protect-dynamodb items as they
are while the paragraph above still required { storedEqlVersion: 2 }.
It now says a top-level scalar item reads either way and the option is
needed for a grouped v2 field declared by its bare leaf name. The advice
to REMOVE a stale __hmac on in-place updates now covers every domain
that writes no __hmac, not only Boolean.
A v2 downcase column mapped to TextEq is hashed case-sensitively by Stack, so a mixed-case write gets an __hmac that a lowercase query never matches. The skill told customers to lowercase only the query value. It now says to lowercase every write too, notes that this drops the original casing from the ciphertext, and suggests a separate attribute for the original when the app needs it. The write-path behaviour is already pinned by the 'downcase on the write path' test.
… its tests Customers on @cipherstash/protect 10.2.1 and @cipherstash/protect-dynamodb 6.0.1 need a written path to Stack 1.x that says plainly what changes and what does not. The guide walks the upgrade as a code-only change over untouched items, and backs every claim with the name of the compat or crossover test that proves it. A guide that cites tests drifts the moment a test is renamed, so protect-upgrade-guide.e2e.test.ts fails when a cited test name no longer exists in its suite, or when the Protect versions the guide names differ from the ones the suites install. It needs no credentials and runs in the default e2e suite; the live suites still prove the claims themselves.
…ew claims The step 4 encryptQuery sample read `term.data.hm`, which does not compile: the result type also covers ciphertexts, strings and null. It now narrows through an `hmacOf` helper. Step 2 now names CS_CTS_HOST and CS_ZEROKMS_HOST (Protect 10.2.1 needed them outside ap-southeast-2, and Stack still honours them), says to carry over a configured keyset, and fixes a "done" check that `protect-ffi` always tripped. The unindexed-column row is limited to text, which is all the suite tests. Tests: fractional DoubleEq round trip, the Protect JSON schema failing on a Stack Json rewrite, the JSON gap under both read options, an IntegerEq assertion the test name already claimed, and a lockfile check on the customer's protect-ffi and schema versions.
`result.data.hm` does not type-check: encryptQuery's result type also covers ciphertexts, strings and null. The stash-dynamodb examples now use an `hmacOf` helper that narrows to the equality term and throws when there is none.
… guide pnpm remove refuses a package the manifest does not declare and then removes nothing, so naming the transitive @cipherstash/schema failed step 2 for any app that only declared protect and protect-dynamodb. The guide test now pins the remove command to those two packages. The fractional DoubleEq compat test also asserts the legacy HMAC is a string, so it cannot pass when both sides return undefined.
The upgrade guide is tied to compatibility tests, but nothing checks that a reader can follow it. The migration harness (#1540) will have an agent upgrade a small Protect.js app using only the guide, judge the result with deterministic checks, and revise the guide from what went wrong. This lands the groundwork the tickets build on: the harness glossary, linked from CONTEXT-MAP.md, and the fixture app that every Run starts from, which stores encrypted customers in DynamoDB and copies three columns into Postgres EQL v3. The Run driver, Oracle and agents follow in #1534-#1539.
The spec review for #1540 moved the Oracle's execution of migrated code out of the harness process, so migrated code cannot read the Oracle, its snapshots or other Runs' credentials. The restricted process that installs and runs it needs a name the tickets can share.
Preserve the reviewed migration harness contract in the repository so stacked implementation work has a stable decision record. Separate agent execution from deterministic evidence and retain the Run and Iteration acceptance seams. Refs #1540
Keep the context architecture decision separate from the implementation plan and align the glossary with both. Define attribution, outcome precedence and enforced Editor boundaries so implementation cannot mistake a blocked access for infrastructure failure or an agent error for guide evidence. Refs #1540
Distinguish denied exploration from proven isolation breaches and specify observable probes, stage outcomes and attribution provenance. Define redacted evidence exports and Editor restrictions so implementation can test the boundaries without claiming semantic guarantees from mechanical checks. Refs #1540
Define unresolved stage outcomes, observable runtime supervision and evidence mapping so implementation does not infer guide failures from setup errors or agent self-reporting. Keep the dated plan and glossary consistent, with explicit redaction checks and acceptance IDs for the child tickets. Refs #1540
Keep the architecture and full implementation contract in the context ADR as requested, rather than splitting authority across a separate plan. Preserve all requirements and acceptance decisions for the stacked implementation work. Refs #1540
Keep each Migrator independent of earlier friction and use one mandatory supervision contract across execution and acceptance. Keep policy in the authoritative ADR, simplify friction identity, and assign isolation proof before application execution. Refs #1540.
Establish the Run seam with explicit stage outcomes and trusted supervision before enabling migration experiments. Add credential-free CI probes so Linux backend and store failures are visible while full Fixture App acceptance remains under review. Refs #1534
Harness CI must declare workspace build dependencies and matching push and PR path filters to satisfy repository guards. Route its checks through uncached Turbo tasks and format the new backend files so repository lint passes. Refs #1534
Make behaviour checks distinguish real writes and copies, initialize assigned stores inside App Host, and reject malformed logs or missing oracle checks. Treat lost observers and exited hosts as terminal failures, separate actor policies, and route assigned stores through explicit container networks. Enable strict checking across the harness and align its documentation and CI inputs. Expanded live isolation and forwarding verification remains pending; no complete credentialed no-op Run is claimed. Refs #1534
Resolve the CLI root inside its checkout so fixture paths and revision capture use the intended repository. Open agent-provided Friction Logs nonblocking before validating their type, preventing FIFOs from hanging the controller outside actor supervision. Cover both failures through the CLI and real FIFO regression seams. Refs #1534
Compare Oracle-owned stored ciphertext and equality terms after read-only application checks and before mutations. Report changed item attributes without disclosing stored values. Refs #1535.
Validate complete Legacy Data snapshots before migration and run unchanged and rewrite-on-read applications through isolated real drivers in credentialed CI. Refs #1535.
|
Contributor
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true |
CI Docker does not publish ports on internal-only bridge networks. Keep trusted disposable stores on dedicated bridges with loopback-only host ports while actor egress stays enforced in its isolated namespace. Refs #1535.
Use the existing Turbo task and loose environment mode for credentialed Run checks so workflow dependency guards pass and credentials reach the live suite. Refs #1535.
Follow the repository test environment convention so documented local env files enable credentialed acceptance cases. Refs #1535.
tobyhede
force-pushed
the
feat/migration-harness-run
branch
from
November 10, 2026 06:25
50cf037 to
899d121
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Detect migrations that rewrite Legacy Data while reading it. Compare Oracle-owned ciphertext and equality terms before mutation checks, and identify changed items and attributes without exporting their stored values.
Changes
Verification
Related
Closes #1535. Stacked on #1576. Followed by #1581.
Review notes
Commits are signed. Separate worktree: .worktree/migration-harness-no-rewrite. Live tests use Docker and CipherStash credentials in the existing E2E job, without model tokens.