Skip to content

feat(e2e): verify the migration Oracle with reference and broken applications - #1581

Draft
tobyhede wants to merge 8 commits into
feat/migration-harness-no-rewritefrom
feat/migration-harness-selftest
Draft

tobyhede wants to merge 8 commits into
feat/migration-harness-no-rewritefrom
feat/migration-harness-selftest

Conversation

@tobyhede

@tobyhede tobyhede commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Prove that the Migration Harness accepts a known-correct Protect.js upgrade and rejects the documented migration mistakes. Deterministic Fake Migrators exercise the real Run pipeline without model calls.

Changes

  • Add a hidden Reference Migration using Stack, normalized downcase writes, decimal-string bigint boundaries and boolean scan/filter lookups.
  • Check legacy packages/imports/endpoint shim removal, resolved Stack/FFI versions and stored boolean-HMAC removal immediately after setActive.
  • Resolve published npm versions or pnpm-packed tree tarballs with SHA-256 metadata.
  • Exercise reference/no-op, broken downcase/bigint/HMAC/rewrite, missing-log, legacy-FFI, tree-tarball and application/lifecycle-denial cases through real Runs.
  • Preserve Fake Migrator artifacts, bound failure diagnostics and provide system CA certificates for the native Stack client.

Verification

  • Local harness typecheck, formatting and self-tests passed: 37 tests passed locally; 13 credentialed cases reserved for CI.
  • Full CI passed at 9693bcd: Node 22/24, Bun, WASM E2E, credentialed E2E, lint, harness isolation/store probes and applicable EQL checks.
  • Credentialed harness matrix: 50 tests passed across 12 files, including all 11 migration variants and the two inherited no-rewrite cases. CI job.
  • Standards and Spec review agents rechecked the identified corrections without remaining concrete findings in their focused scope.
  • Every commit, including merge commits, is signed and locally verified.

Related

Closes #1536. Stacked on #1580, then #1576.

Review notes

Separate worktree: .worktree/migration-harness-selftest. The Reference Migration remains outside actor mounts. Tests use the existing credentialed E2E job with no model tokens. Initial CI found missing CA certificates in the slim image; the final live matrix proves the correction.

Run deterministic Fake Migrators against the complete Oracle, inspect stored boolean terms immediately after updates, and resolve npm or tree tarballs before application execution. Refs #1536.
Pack workspace sources through pnpm, retain Fake Migrator artifacts, and exercise caught createApp and npm lifecycle denials through the real Run pipeline. Refs #1536.
@coderabbitai

coderabbitai Bot commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

@changeset-bot

changeset-bot Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 9693bcd

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Explain published and tree Stack sources and the credentialed deterministic matrix so contributors can reproduce the CI acceptance Runs. Refs #1536.
Report only stage failures and failed Oracle checks so GitHub issue matchers can process failures. Stop expensive later Runs after the first failing case. Refs #1536.
Stack native clients load system CA certificates when constructing TLS clients. The slim Node image lacked the trust bundle, so every Reference Migration exited before Oracle calls. Refs #1536.
Use the real per-invocation synthetic protected resource for application and lifecycle denial cases and require a forbidden decision from the trusted observer. Refs #1536.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant