Skip to content

Add explicit trust options to investigation run start - #461

Merged
maximelb merged 20 commits into
masterfrom
w10/explicit-investigation-trust
Nov 14, 2026
Merged

maximelb merged 20 commits into
masterfrom
w10/explicit-investigation-trust

Conversation

@maximelb

@maximelb maximelb commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Adds repeatable investigate runs start --trust-lookup NAME and --trust-value VALUE options. They add to a template's trust for one run only. Automatic runs use template trust only; manual --auto still selects a template and accepts run-only additions. The identity starting the run needs lookup.get for referenced Lookups; Autopilot uses its onboarding key. No new settings/read permission is introduced.

The SDK validates request trust before sending: an object containing only lookups/values lists of nonblank strings, at most64Lookup names and100inline values. Unknown keys, wrong types, null/blank items and over-limit lists raise ValidationError. None, empty objects and all-empty lists omit trust entirely; otherwise only nonempty keys are sent. Normalization preserves caller input. Request-level trust remains unavailable on waves.

Base is master after the investigation CLI landed at6b43924157d562307c50ccccb55ae41adb397742. The integration is additive. All master SDK/CLI test functions, bounded pagination/cycle detection, configurable follow timeout and last-delivered resume hints, canonical fixture generation/public schemas and explicit UTF-8 text I/O are preserved. The diff against master is six trust-specific files. CHANGELOG, command synopsis and --ai-help document both flags and run-only semantics.

Validation on this exact source: focused SDK/CLI/discovery/command-map/docs/fixture tests734PASS. Full documented CI command python -m pytest tests/unit/ tests/microbenchmarks/ --benchmark-disable -v --reruns 2 --reruns-delay 5 -rs PASS6377tests/33intentional guards/21warnings on CPython3.14.10 with both real compression.zstd and zstandard0.25.0. All93microbenchmarks passed. Guards are24no-decoder scenarios,4decoder-available scenarios,2existing docstring checks and3platform checks; no unavailable-codec skips or weakened/excluded assertions. Tests cover invalid shapes, exact limits, minimal wire bodies, caller immutability and help. Diff/hygiene checks PASS.

Maintainer owns refreshed review, CI monitoring and merge. No tag or release is created.

maximelb and others added 16 commits November 10, 2026 13:11
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
YAML 1.2 number handling for template sources, iter_actions cursor guard,
JSON-lines stream for runs start --follow, landed-write guard for template
saves, hive permission hint for template records.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…d tags

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… every failure, run mode docs, example check

- wave and --samples-file are read with YAML 1.2 scalar rules (off/yes stay text, dates stay strings)
- unreadable or malformed input files and every Hive/API failure render as structured errors
- help and docs: the run mode default is full; triage is cheaper
- docs wave example selects the real ioc.prevalence field (seen); a test checks every documented wave against a recorded operator catalogue

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
- runs start --urgency is low|medium|high|critical (default derived from the alert, else medium)
- --auto sends no plan (the engine refuses plan.auto for a person's run) so the engine picks the active template by applies_to; new --category/--severity/--priority/--tag carry the alert attributes it matches on
- templates test --strict help: refuses overrides that would loosen the policy
- runs list --trigger takes the trigger kind, --verdict also takes undecided, --started-by also matches a name
- policy get --template; --case-mode and --wait help corrected

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…h shapes

Engine v0.1.4 lets a template's report.summary sentence be a string or
{text, when}. The SDK and CLI treat a template as an opaque mapping, so no code
change is needed; tests now pin that save, get, install-defaults and the YAML
round trip keep both shapes unchanged.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…L parser does

Unquoted scalars under a fact's args follow gopkg.in/yaml.v3 (012, 1_000, 0o17,
1e3, timestamps), other template fields keep the author's text except the
numeric and boolean ones, mapping keys are text. A fixture generated from the
engine's own parser pins every edge spelling. The template dumper quotes every
string the engine would read as something else, and save/promote validate text
proven to read back as the stored record. NaN/Infinity are refused as
invalid_input. The docs state who a run executes as when the policy names a
secret identity.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…s with a generated fixture

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Expose DELETE for runs with validated IDs, explicit CLI confirmation, discovery and documentation. Preserve API errors and disable gateway timeout retries for deletion writes.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… a server setting name from the docs

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…carries one hint

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* Preserve safe template test diagnostics in compact and tabular output

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Keep template test diagnostics consistent across output formats

Preserve the priced subtotal, share diagnostic filtering, append CSV columns, and document the output beside templates test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Show an unknown LLM cost as blank, never as 0

A test result from a service that reports no cost object now leaves the llm_cost cell empty and
prints "LLM cost: unknown", so it cannot be read as free.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@maximelb

Copy link
Copy Markdown
Contributor Author

/lc-review

@limacharlie-event-notifications limacharlie-event-notifications Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI Code Review Summary

No high or medium issues. The new options pass through correctly, and the test calls the real command and checks the request sent to the SDK. Two low-priority notes on the docs are inline.

General Feedback

  • No test-time-budget check run was found for this commit. The base branch feature/ie-cli has no branch protection or ruleset that requires it, so it does not block this review.
  • The CLI does not check the documented limit of 100 trust values, and it does not check for empty strings. The PR body says the engine enforces these rules. This is fine, but a bad value will only show up as an engine error.

Blast Radius

  • This only adds optional fields. investigate runs start sends a trust object only when one of the new flags is given, so current callers send the same request as before.
  • Investigation.start_run only had its docstring changed. Its public SDK behaviour is the same.

This review was generated automatically by Claude Code.

Comment thread doc/cli/investigation-engine.md Resolved
Comment thread doc/cli/investigation-engine.md Resolved
lcbill
lcbill previously approved these changes Nov 14, 2026
@maximelb

Copy link
Copy Markdown
Contributor Author

/lc-review

@limacharlie-event-notifications limacharlie-event-notifications Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI Code Review Summary

Re-reviewed at the new commit, which drops trust: None from start_run requests. The change is correct. It copies the request before removing the key, so the caller's dict is not changed. The new test calls the real start_run, checks the JSON that is sent, and checks that the caller's object is unchanged. No high or medium issues.

General Feedback

  • The two low-priority doc notes from the last review still apply. doc/cli/investigation-engine.md has not changed since then.
  • No test-time-budget check run was found for this commit. The base branch feature/ie-cli does not require this check, either in branch protection or in rulesets.

Blast Radius

  • Investigation.start_run (public SDK) now removes a trust key whose value is None before it sends the request. Callers that passed trust=None used to send \"trust\": null and now send nothing. All other requests are sent the same as before.
  • investigate runs start sends trust only when --trust-lookup or --trust-value is given.

This review was generated automatically by Claude Code.

@maximelb

Copy link
Copy Markdown
Contributor Author

/lc-review

@limacharlie-event-notifications limacharlie-event-notifications Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI Code Review Summary

Re-reviewed at 406f803. The new commit makes Investigation.run_wave refuse a request that has a trust key, and adds tests for it. It also adds a test that test_plan sends no trust. No high or medium issues. The two low-priority doc notes from earlier reviews still apply.

General Feedback

  • run_wave checks "trust" in request, so {"trust": None} is refused too. start_run drops trust: None silently. This is a small inconsistency. If a null value should be accepted the same way in both, check request.get("trust") is not None instead. (Low)
  • test_plan_does_not_add_request_trust gives little protection: test_plan has no trust parameter and builds its own body, so this test cannot fail unless someone adds that code later. (Low)
  • doc/cli/investigation-engine.md has not changed since the earlier reviews. Both open doc threads still apply.
  • No test-time-budget check run was found for this commit. The base branch feature/ie-cli does not require this check in branch protection or in rulesets.

Blast Radius

  • Investigation.run_wave (public SDK) and investigate wave now raise ValidationError when the request or wave file has a top-level trust key. Before, the key was sent to the engine. Callers or wave files that include trust will now fail on the client side.
  • Investigation.start_run drops trust: None and does not change the caller's dict. Other requests are sent the same as before.
  • investigate runs start sends trust only when --trust-lookup or --trust-value is given.

This review was generated automatically by Claude Code.

@maximelb

Copy link
Copy Markdown
Contributor Author

/lc-review

@limacharlie-event-notifications limacharlie-event-notifications Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI Code Review Summary

Re-reviewed at 388a446. This commit moves the Explicit trust section before ## See Also, makes it a ## heading, and rewords the automatic-runs sentence. It also removes test_plan_does_not_add_request_trust. No code changed in this commit. No high or medium issues.

General Feedback

  • The automatic-runs sentence is still unclear (Low). "Automatic runs use only the template's trust" can still be read as covering runs start --auto. But --auto --trust-lookup X does send trust (investigate.py lines 974-975). If the sentence means runs started by D&R rules, say so.
  • run_wave refuses any trust key, even None, while start_run silently drops trust: None. This small difference is still there (Low, raised in the last review).
  • No test-time-budget check run was found for this commit. The base branch feature/ie-cli does not require this check in branch protection or in rulesets.

Blast Radius

  • Investigation.run_wave (public SDK) and investigate wave now raise ValidationError on the client side when the request or wave file has a top-level trust key. Before, the key was sent to the engine.
  • Investigation.start_run drops trust: None from a copy of the request, so the caller's dict is not changed. Other requests are sent as before.
  • investigate runs start adds trust only when --trust-lookup or --trust-value is given. Existing invocations send the same request as before.

This review was generated automatically by Claude Code.

@maximelb

Copy link
Copy Markdown
Contributor Author

/lc-review

@maximelb
maximelb force-pushed the w10/explicit-investigation-trust branch from 388a446 to 1481d6a Compare November 14, 2026 15:45

@limacharlie-event-notifications limacharlie-event-notifications Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI Code Review Summary

Re-reviewed at 1481d6a. Compared with 388a446, this commit only rewords the Explicit trust doc section. It now says that runs started by D&R rules or Autopilot use only the template's trust, and that the manual --auto option still accepts the run-only additions. That matches the code (investigate.py lines 974-975). No high or medium issues.

General Feedback

  • run_wave refuses any trust key, even None, while start_run silently drops trust: None. This small difference is still there (Low, raised before).
  • doc/cli/investigation-engine.md has two blank lines before ## See Also (cosmetic).
  • No test-time-budget check run was found for this commit. The base branch feature/ie-cli does not require this check in branch protection or in rulesets.

Blast Radius

  • Investigation.run_wave (public SDK) and investigate wave now raise ValidationError on the client side when the request or wave file has a top-level trust key. Before, the key was sent to the engine.
  • Investigation.start_run drops trust: None from a copy of the request, so the caller's dict is not changed. Other requests are sent as before.
  • investigate runs start adds trust only when --trust-lookup or --trust-value is given. Existing invocations send the same request as before.

This review was generated automatically by Claude Code.

Base automatically changed from feature/ie-cli to master November 14, 2026 23:24
@maximelb
maximelb dismissed lcbill’s stale review November 14, 2026 23:24

The base branch was changed.

# Conflicts:
#	CHANGELOG.md
#	doc/cli/investigation-engine.md
#	limacharlie/commands/investigate.py
#	limacharlie/sdk/investigation.py
#	tests/unit/fixtures/ie_catalogue.json
#	tests/unit/fixtures/investigation_scalars.json
#	tests/unit/fixtures/investigation_typed_fields.json
#	tests/unit/test_cli_investigate.py
#	tests/unit/test_investigation_examples.py
#	tests/unit/test_sdk_investigation.py
@maximelb

Copy link
Copy Markdown
Contributor Author

/lc-review

@limacharlie-event-notifications limacharlie-event-notifications Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI Code Review Summary

Re-reviewed at d8eb5c1, now based on master. I found no high or medium issues. start_run checks the trust object before it sends anything: the type, the allowed keys, the list types, blank entries and the 64/100 limits. It copies the request before changing it, so the caller's object stays the same. The CLI turns a ValidationError into an error document through _engine_errors, because ValidationError is a LimaCharlieError. The SDK tests call the real start_run and check the JSON body that is sent. They cover the invalid shapes, the exact limits, the empty forms and the immutability of the caller's object.

General Feedback

  • The CLI tests mock the SDK. No test runs runs start --trust-value "" (or 65 --trust-lookup flags) through the real start_run to show that the user gets a clean error before any request is sent. The SDK tests cover the validation itself, so this is a small gap (Low).
  • run_wave refuses any trust key, even None, while start_run silently drops trust: None. This small difference is still there (Low, raised before).
  • doc/cli/investigation-engine.md has two blank lines before ## See Also (cosmetic).
  • No test-time-budget check run was found for this commit. The base branch master does not require this check in branch protection or in rulesets, so it does not block this review.

Blast Radius

  • Investigation.start_run (public SDK) now raises ValidationError on the client side when a request has a malformed trust value. It drops None, empty objects and empty lists, and sends only the non-empty keys. Requests without trust are sent the same as before.
  • Investigation.run_wave and investigate wave now refuse any top-level trust key on the client side. Before, the key was sent to the engine.
  • investigate runs start adds trust only when --trust-lookup or --trust-value is given. Existing invocations send the same request as before.

This review was generated automatically by Claude Code.

@maximelb
maximelb merged commit 01120bb into master Nov 14, 2026
7 checks passed
@maximelb
maximelb deleted the w10/explicit-investigation-trust branch November 14, 2026 23:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants