Skip to content

[0.38] #3701 (2/2) cleanup deletes only with deployment evidence, never a lockfile hash alone - #4049

Draft
Daniel Meppiel (danielmeppiel) wants to merge 13 commits into
danielmeppiel-literate-broccolifrom
danielmeppiel-3701-enforcement
Draft

Daniel Meppiel (danielmeppiel) wants to merge 13 commits into
danielmeppiel-literate-broccolifrom
danielmeppiel-3701-enforcement

Conversation

@danielmeppiel

@danielmeppiel Daniel Meppiel (danielmeppiel) commented Nov 20, 2026 •

Copy link
Copy Markdown
Collaborator

Description

Draft, PR 2 of 2 for #3701 (enforcement and render fallback). Stacked on #4023 (receipt plumbing).

A lockfile hash alone no longer authorizes deleting a deployed file. Every lockfile-driven deletion now needs evidence that APM deployed the current bytes. That covers install and update orphan and stale cleanup, target contraction, uninstall (Step 3 and Phase 1), prune, the deferred skill-directory pass, integrator sync and skill path migration. The evidence is one of:

  • a matching local deployment receipt ([0.38] #3701 (1/2) record deployment receipts for every file primitive #4023 records one for every final-rendered write), or
  • byte equality with the locked render: the drift replay (install/drift.run_replay, the existing package-image owner) renders the pre-transition lock into a private scratch tree, with the targets the lock recorded as well as the manifest's, so a just-dropped target stays provable. It runs only when some candidate has no matching receipt, fetches the locked commit when the checkout is gone, and falls back to rendering each owning package alone when the whole-lock render fails, so one unmaterializable package loses only its own evidence.

A stale receipt does not block the render comparison: receipt OR render. A file adopted from a newer render after a pull is still removable.

Without evidence the file is kept, and a warning names it and the manual fix. A file that differs from its evidence is kept as user-edited. cowork:// rows lose their exemption. Managed final symlinks keep their current unlink rule. Uninstall still stops, keeping the declaration, when a target file is retained.

One owner, closed-world

flowchart LR
  I[install / update] --> E[DeploymentEvidence]
  U[uninstall] -->|prepare before removing sources| E
  P[prune] -->|prepare before removing sources| E
  M[skill path migration] --> E
  E --> R[remove_stale_deployed_files]
  S[BaseIntegrator / hook / skill sync] --> RS[remove_synced_files] --> R
  E -->|receipts| L[DeploymentMaterializationSession]
  E -->|locked render| D[drift.run_replay]
Loading
  • integration/cleanup.py owns DeploymentEvidence (receipts plus the lazy locked render) and remove_stale_deployed_files(..., evidence=...). evidence is keyword-only with no default, so no caller can fail open.
  • remove_synced_files is the only path for integrator sync deletions. Hook and skill sync_integration and BaseIntegrator.sync_remove_files used to unlink/rmtree with no check.
  • install/phases/cleanup.deployment_evidence(ctx) builds one evidence per install, from ctx.existing_lockfile.
  • Uninstall and prune call DeploymentEvidence.prepare before removing package sources.
  • Architecture guard install-deployment-provenance-state now requires these routes, with mutation traps.

Known limitations and consequences

  1. cowork:// has no render fallback. The drift replay does not render experimental external targets, so a receipt-less cowork row is kept and named. A matching receipt still removes it. Follow-up: teach the replay to project copilot-cowork.
  2. Legacy sidecars from older APM versions are no longer auto-removed. For example, the Copilot hook package.json an older APM wrote carries only a lockfile hash, and today's locked render does not produce it. Install and update now keep it and name it, and uninstall stops naming it. The manual fix is to delete it and rerun. test_hook_js_sidecar_lifecycle_contract.py now asserts this flow.

Decision for Daniel

  • Amendment (proposed 0.1.76, commit spec(normative, pending Daniel)) adds req-lk-024 (Section 5.2, consumer MUST): a lockfile-driven deletion needs a matching local deployment record, or byte equality with the locked version's render. A lockfile hash alone never authorizes it. Unproven or differing files are kept and named.
  • Alternative: keep the rule implementation-only. Drop that one commit (it touches only the spec, its manifest, one conformance test and CONFORMANCE), add an apm-spec-waiver: with an informative note in Section 10.7.
  • Recommendation: adopt 0.1.76. Deletion authority is consumer-observable, and other implementations reading a committed lockfile face the same forged-row risk. Section 10.7 already says containment "does not establish ownership or authorize deletion"; this makes that testable.

Spec renumbering at merge

0.1.76 is provisional. Recompute the amendment number, the statement count (now 149, of which 144 MUST and 5 SHOULD) and the Appendix D row against main at merge time, then regenerate CONFORMANCE.{md,json}.

Docs re-home after #3996

Edited here (current main layout) After #3996
enterprise/security.md, "Lockfile cleanup authority" paragraph #3996 rewrites this page and drops the paragraph. Re-home it as one control in the rewritten enterprise/security.md (or under "Drift and integrity" in reference/lockfile-spec.md).
reference/cli/install.md, "Stale-file cleanup" bullet Same file and bullet still exist; reapply.
reference/cli/update.md, receipt-less update paragraph Same file; reapply.
reference/cli/uninstall.md, retained-paths paragraph Same file; reapply.
reference/cli/prune.md, cleanup paragraph Same file; reapply.
packages/apm-guide/.apm/skills/apm-usage/commands.md Unchanged by #3996; reapply.

Review folds (c9b13584 on babed8a)

Finding Fix Regression test (red on babed8a, green now)
HIGH: a stale receipt blocks render evidence verdict falls through to the render on a receipt mismatch test_adopted_after_pull_then_uninstall_uses_the_locked_render
MEDIUM: a dropped target is never provable without receipts; warning printed twice Render with lock-recorded targets (resolve_audit_targets(include_recorded=True) via ReplayConfig.include_recorded_targets); duplicate inline lines removed; apm update renders install diagnostics test_dropped_target_is_proven_by_the_render_of_recorded_targets; update test asserts the kept file is named once
MEDIUM: all-or-nothing render; silent uninstall abort Per-package fallback render naming the failed package; uninstall flushes diagnostics before it stops test_render_evidence_is_per_package[other-package], [removed-package]
Upgrade note for legacy sidecars One CHANGELOG upgrade-notes line (moved from Fixed) -
LOW: user-scope syncs retired receipts in the project store DeploymentEvidence is scope-bound; remove_synced_files passes its user_scope test_synced_removal_retires_receipts_in_the_evidence_scope

Pre-merge

Issue and approved scope

Issue: Fixes #3701

Human scope-approval comment: #3701 (comment) (apm-scope:v1, record-present via scripts/governance/eligibility.cjs)

Does this PR complete the issue? Yes, together with #4023 (its base).

Type of change

  • Bug fix
  • New feature
  • Documentation
  • Maintenance / refactor

Testing

Local CI-mirror evidence. CI does not run on this stacked PR, and the sandbox has 2 CPUs.

Check Command Result
Lint uv run --extra dev ruff check and ruff format --check on src/ tests/ scripts/... clean
Duplication pylint --enable=R0801 --min-similarity-lines=10 10.00/10
Architecture bash scripts/lint-architecture-boundaries.sh; tests/integration/test_architecture_materialization_receipts.py exit 0; 34 passed
Auth signals bash scripts/lint-auth-signals.sh clean
File length no covered file over 2100 lines ok
Unit pytest tests/unit -n 3 (excluding the sandbox-environment files that also fail on main) 27869 passed, 0 failed
New lifecycle tests APM_BINARY_PATH=.venv/bin/apm pytest tests/integration/test_cleanup_deployment_evidence_lifecycle.py 14 passed (4 added for review c9b13584, red on babed8a)
Lifecycle smoke (CI subset) APM_E2E_TESTS=1 pytest -m 'lifecycle_smoke and not lifecycle_merge_group' tests/integration 784 passed, 24 failed. 20 also fail on pristine origin/main in this sandbox (9 test_marketplace_explicit_refresh_lifecycle, 10 test_unix_installer_path_integration, 1 test_hook_event_native_lifecycle::test_claude_project_hook_runs_from_external_cwd). The other 4 were real and are fixed (test_hook_js_sidecar_lifecycle_contract x3, test_stale_cleanup_hash_authority_lifecycle x1): rerun 6 passed.
Spec conformance pytest tests/spec_conformance; python -m tests.spec_conformance.orphan_check 629 passed and 3 failed; the 3 were fixed (rerun 60 passed); orphan_check OK (149 aligned)

The new lifecycle tests run on committed-output clones with a fresh HOME and APM_HOME, and assert that no receipts exist before each command. They cover:

  • update (locked-render bytes removed, an edited file kept, a forged row kept and named);
  • install orphan cleanup;
  • uninstall with apm_modules present (source unreachable), with only the source reachable, and with neither (exits 1 naming the files, declaration kept);
  • prune (render taken before the sources are removed);
  • the deploying machine (receipts remove instruction, prompt, agent and hook files with the source gone);
  • cowork:// rows (user scope, experimental flag, APM_COPILOT_COWORK_SKILLS_DIR fixture: a receipt removes the file; receipt-less or edited files are kept and named).

Changelog

  • One line under ## [Unreleased] / Fixed, ending (closes #3701, #4049).

Spec conformance (OpenAPM v0.1)

  • Spec edit: req-lk-024 anchor, prose and Appendix C row (proposed 0.1.76).
  • Manifest edit: openapm-v0.1.requirements.yml.
  • Test edit: tests/spec_conformance/test_deletion_evidence_reqs.py (@pytest.mark.req("req-lk-024")).
  • CONFORMANCE.{md,json} regenerated.

… hash alone (#3701)

remove_stale_deployed_files now requires a DeploymentEvidence (keyword,
no default): a matching local deployment receipt, or byte equality with
the drift replay of the pre-transition lock (the locked render). Files
that differ are kept as edited; files without evidence are kept with a
warning naming them and the manual fix. cowork:// rows lose their
exemption. Integrator sync (base, hooks, skills), the deferred skill
directory pass, skill path migration, install, update, uninstall and
prune all route through this owner; uninstall and prune render before
removing package sources. Managed final symlinks keep their rule.

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
…en files inline (#3701)

Committed-output clones with fresh HOME/APM_HOME and no receipts cover
install orphan cleanup, update stale cleanup, uninstall (modules present,
source only, neither) and prune: locked-render bytes go, a forged row's
user file and an edited file stay. The deploying machine uninstalls every
primitive through receipts with the source gone. cowork:// rows go only
with a matching receipt. Install and update now name kept unproven files
inline, as they do user edits.

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
…ment evidence (#3701)

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
#3701)

Proposed amendment 0.1.76 (provisional number; counts recomputed at
merge). Adds req-lk-024 (Section 5.2, consumer MUST): a lockfile-driven
deletion of a deployed file needs a matching local deployment record or
byte equality with the locked version's render; a lockfile hash alone
never authorizes it; unproven or differing files are kept and named; the
render is established before the source is removed. Updates Sections
1.3, 5.7, 10.7, 10.11, 11.3.2, Appendix C/D, the requirements manifest,
a req-lk-024 conformance test and CONFORMANCE.{md,json}. Isolated so it
can be dropped if the editor keeps the rule implementation-only.

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
…vidence (#3701)

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
…kill dirs (#3701)

Unit tests now state their evidence explicitly: receipted files on the
deploying machine, or none on a clone. Tests whose point was that a
lockfile row alone must not delete keep that intent. Legacy deployed_skills
migrations record skill directories with a trailing slash; the cleanup
owner now recognizes them instead of refusing the entry. Documents the
pipeline.py and manifest_reconcile.py LOC budgets the required evidence
argument costs.

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
…follow the evidence rule (#3701)

Uninstall target cleanup now names each file kept for lack of evidence, as
install does. Hook sidecars an older APM wrote carry only a lockfile hash, so
their lifecycle contract now expects them kept and named, with the manual
fix (delete, then rerun) restoring the contract.

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
…fallback (#3701)

Review folds (c9b13584 on babed8a):
- A stale receipt no longer blocks the render comparison: a file adopted
  from a newer render after a pull is removed when it equals the locked
  render (EDITED only when no evidence matches).
- The locked render also covers targets the lock recorded, so a target
  dropped from apm.yml stays provable on a clone without receipts.
- When the whole-lock render fails, each owning package renders alone, so
  one unmaterializable package loses only its own evidence; the failure is
  named per package.
- Kept files are named once: uninstall flushes its diagnostics before it
  stops and with its outcome, apm update renders install diagnostics, and
  the duplicate inline lines are gone.

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
…decars (#3701)

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
#3701)

DeploymentEvidence is scope-bound, so remove_synced_files passes its
user_scope through: user-scope uninstall syncs validate user roots and
retire receipts in the user store instead of the project one.

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Install, update, uninstall and prune references, the enterprise security
cleanup-authority paragraph and the apm-usage commands resource now say a
lockfile hash alone never authorizes deleting a deployed file.

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant