Skip to content

test: re-baseline evidence pins for the PyAutoArray#683 pseudo-determinant #822

Closed

Description

@Jammy2211

Overview

Heart is RED. Since 2026-11-09 10:59, every PyAutoGalaxy and PyAutoLens main Tests run has failed (3.12, 3.13, nojax). The failing tests are figure-of-merit / log-evidence pins for fits with an intrinsic-regularized pixelization or basis. The cause is PyAutoArray#683 (PyAutoArray#682), which deliberately normalises log det H with the pseudo-determinant for Constant, ConstantSplit, Adapt, AdaptPower, AdaptSplit and AdaptSplitPower. That raises log evidence by +½·ln(1e8) = +9.21034 nats per intrinsic-regularized mapper. The maintainer approved the change on 2026-11-09. PyAutoArray#683's companion PR autolens_workspace_test#358 re-pinned the workspace values, but nothing updated the downstream PyAutoGalaxy/PyAutoLens unit-test pins. This task re-baselines them. No source changes.

Bisect evidence (PyAutoArray, against the 11 failing PyAutoGalaxy tests)

PyAutoArray result
e2b05f0 (before PyAutoArray#681 and PyAutoArray#683) 108 passed
c020cc2 (PyAutoArray#681 merged) 108 passed, so PyAutoArray#681 is not the cause
6ae33d4 (PyAutoArray#683 merged) 11 failed
1482dd9 / af80e69 / 2e494f9 (main) 11 failed
main with a589c35 (PyAutoArray#683) reverted 108 passed

The same 16 PyAutoLens tests fail on PyAutoArray main locally, and pass at c020cc2 (87 passed in the four affected files).

Justification of every new number

Each new pin is the measured value on PyAutoArray main. For each failing assertion I recorded figure_of_merit at c020cc2 (old) and at main (new):

  • Single intrinsic mapper: Δ = +9.210340 (= ½·ln(1e8) = 9.2103404). Cases with Δ = 9.21058 or 9.210342 carry round-off from the old jitter-determined term, which PyAutoArray#683 removes, as described in its PR body.
  • Two intrinsic mappers (test__simulate_imaging_data_and_fit__known_likelihood and test__simulate_interferometer_data_and_fit__known_likelihood, two pixelized source planes): Δ = +18.420685 = ln(1e8).
  • Analytic check, for the PyAutoGalaxy standard_galaxies__known_figure_of_merit setup (16×16 RectangularUniform, Constant(1.0)): λ_min(H) = 1.0000e-8 (the jitter) with the next eigenvalue at 0.0384. The eigen oracle gives ½(log pdet − log det) = 9.210340, and the code's new term (275.11594) matches the oracle pdet (275.11593).
  • One exception, linear_light_profiles_and_pixelization__delaunay_split (Delaunay(25, zeroed_pixels=5) + AdaptSplit(0.01, 0.1)): Δ = +9.5266. In this weakly regularized case the jitter-free H₀ has an exact null mode (row sums 1e-21) but a second eigenvalue of only 1.4e-9, below the 1e-8 jitter. Grounding H = H₀ + εI is then not exactly the pseudo-determinant (pdet(H₀) itself gives −358.72; the code gives −354.56). The new value is deterministic and is exactly what PyAutoArray#683 defines (P·det(grounded H)), so it is re-pinned as measured. The ε ≳ λ₂ regime is flagged as a possible follow-up on PyAutoArray#682's semantics. It is not a regression in this task.

Plan

  • Re-pin the 16 failing PyAutoLens test assertions (plus any follow-on assertions in the same tests that were masked by the first failure) to the measured values on PyAutoArray main. Tolerances unchanged.
  • Files: test_autolens/{imaging,interferometer}/test_{fit,simulate_and_fit}_*.py (21 pins, 4 files).
  • Validation: full PyAutoLens suite against PyAutoArray main locally, then exact-head CI on all legs.
  • Tier: judge — merge mode: human /prm.

Corrective-PR exception for Heart RED

  • Exact Heart RED reason (verbatim from the current Heart feed): PyAutoLens/Tests: failure
  • Causal mapping: reason → this issue → the plan above (re-pin pins shifted by PyAutoArray#683's approved likelihood change) → a test-only diff in this repo.
  • Authorization: live human instruction in session https://claude.ai/code/session_016nhQ8R6PYVh6FagcqYiCNh, recorded in the comment below. The human chose option (b) "re-baseline" after the bisect evidence above was presented.
  • Sibling RED reasons stay open until they clear on their own path (the other library's Tests, Hands Python Version Matrix, Workspace Smoke / validation, worktree drift). This PR does not claim them.

Activity

  1. Jammy2211 commented on Nov 10, 2026

    @Jammy2211
    CollaboratorAuthor

    Corrective-PR exception: authorization record (Heart RED)

    RED reason (verbatim, current Heart feed, 2026-11-10): PyAutoLens/Tests: failure

    Human authorization (live, this incident). Quoted from the maintainer's instruction in session https://claude.ai/code/session_016nhQ8R6PYVh6FagcqYiCNh:

    "Red reasons: PyAutoGalaxy/Tests failure; PyAutoLens/Tests failure; … This is a causal fix for the RED reason, so use the AUTONOMY.md "Corrective-PR exception for Heart RED", not the development override."

    After the bisect evidence (above) was presented, the maintainer chose "(b) Re-baseline": update the expected values in the PyAutoGalaxy and PyAutoLens tests to the measured new values.

    Scope permitted: commit, push, and one pending-release PR whose diff is limited to the 21 test pins in this issue's plan. Merge, close and release stay human (/prm).


    Generated by Claude Code

  2. Jammy2211 commented on Nov 10, 2026

    @Jammy2211
    CollaboratorAuthor

    Shipped — 2026-11-10


    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions