Skip to content

Add org investigation pack authoring and composition guidance - #462

Merged
maximelb merged 24 commits into
masterfrom
feat/w11-surfaces
Nov 14, 2026
Merged

maximelb merged 24 commits into
masterfrom
feat/w11-surfaces

Conversation

@maximelb

@maximelb maximelb commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Add org-authored fact pack get/save/delete helpers and CLI commands using the existing Hive surface. Writes preserve exact YAML, create only by default, and replace only with an explicit etag. Compact discovery retains version, digest and trust metadata.

Document version-required, digest-optional template composition. Preserve resolved_packs, child and trust metadata through SDK responses and CLI JSON. Tests verify source forwarding and response preservation; composition and digest enforcement remain server responsibilities.

Merged current master additively and resolved duplicate-content conflicts while preserving trust validation, UTF-8 I/O, pagination cycle guards and canonical compatibility fixtures. The PR targets master.

Validation on Python 3.14.10: full tests/unit/ and tests/microbenchmarks/ correctness suite with orjson 3.13.0 and again with the stdlib JSON fallback. orjson run: 6389 passed, 33 existing skips. Stdlib JSON run: 6388 passed, 34 existing skips (the existing orjson-availability packaging test is inapplicable without orjson). No failures or reruns. Compression coverage exercises gzip and both Zstandard implementations: compression.zstd (libzstd 1.5.7) and zstandard 0.25.0. No tests weakened or skips added.

Ready for owner review and merge once CI and review complete.

maximelb and others added 20 commits November 10, 2026 13:11
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
YAML 1.2 number handling for template sources, iter_actions cursor guard,
JSON-lines stream for runs start --follow, landed-write guard for template
saves, hive permission hint for template records.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…d tags

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… every failure, run mode docs, example check

- wave and --samples-file are read with YAML 1.2 scalar rules (off/yes stay text, dates stay strings)
- unreadable or malformed input files and every Hive/API failure render as structured errors
- help and docs: the run mode default is full; triage is cheaper
- docs wave example selects the real ioc.prevalence field (seen); a test checks every documented wave against a recorded operator catalogue

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
- runs start --urgency is low|medium|high|critical (default derived from the alert, else medium)
- --auto sends no plan (the engine refuses plan.auto for a person's run) so the engine picks the active template by applies_to; new --category/--severity/--priority/--tag carry the alert attributes it matches on
- templates test --strict help: refuses overrides that would loosen the policy
- runs list --trigger takes the trigger kind, --verdict also takes undecided, --started-by also matches a name
- policy get --template; --case-mode and --wait help corrected

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…h shapes

Engine v0.1.4 lets a template's report.summary sentence be a string or
{text, when}. The SDK and CLI treat a template as an opaque mapping, so no code
change is needed; tests now pin that save, get, install-defaults and the YAML
round trip keep both shapes unchanged.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…L parser does

Unquoted scalars under a fact's args follow gopkg.in/yaml.v3 (012, 1_000, 0o17,
1e3, timestamps), other template fields keep the author's text except the
numeric and boolean ones, mapping keys are text. A fixture generated from the
engine's own parser pins every edge spelling. The template dumper quotes every
string the engine would read as something else, and save/promote validate text
proven to read back as the stored record. NaN/Infinity are refused as
invalid_input. The docs state who a run executes as when the policy names a
secret identity.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…s with a generated fixture

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Expose DELETE for runs with validated IDs, explicit CLI confirmation, discovery and documentation. Preserve API errors and disable gateway timeout retries for deletion writes.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… a server setting name from the docs

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…carries one hint

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* Preserve safe template test diagnostics in compact and tabular output

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Keep template test diagnostics consistent across output formats

Preserve the priced subtotal, share diagnostic filtering, append CSV columns, and document the output beside templates test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Show an unknown LLM cost as blank, never as 0

A test result from a service that reports no cost object now leaves the llm_cost cell empty and
prints "LLM cost: unknown", so it cannot be read as free.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@maximelb

Copy link
Copy Markdown
Contributor Author

/lc-review

@limacharlie-event-notifications limacharlie-event-notifications Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI Code Review Summary

No high or medium issues. Three low-priority notes are inline.

General Feedback

  • test-time-budget: no check run was found on this commit. The base branch w10/explicit-investigation-trust has no required status checks, so this does not block.
  • Missing CLI coverage: investigate pack get and pack save without --etag have no CLI tests. The create-only path is covered only at the SDK level.
  • Packs listing: the test for packs --brief checks only version. digest, source, trust and trust_provenance pass-through is not tested.

Blast Radius

  • New SDK methods Investigation.get_pack, save_pack and delete_pack, plus the new CLI group limacharlie investigate pack. Both are additive.
  • investigate packs --brief output now has extra keys (version, digest, source, trust, trust_provenance) when the server sends them. Strict JSON or CSV consumers of this output can see new columns or fields.
  • The discovery profile investigation_engine lists three more commands. This changes what profile-based command discovery shows.

This review was generated automatically by Claude Code.

Comment thread limacharlie/sdk/investigation.py Resolved
Comment thread tests/unit/test_sdk_investigation.py Resolved
Comment thread doc/cli/investigation-engine.md Resolved
lcbill
lcbill previously approved these changes Nov 14, 2026
@maximelb

Copy link
Copy Markdown
Contributor Author

/lc-review

@limacharlie-event-notifications limacharlie-event-notifications Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI Code Review Summary

No high or medium issues found at this commit. The three earlier low-priority threads are resolved: the name type guard, the renamed composition test, and the docs heading level.

General Feedback

  • test-time-budget: no check run was found on this commit. The base branch w10/explicit-investigation-trust returned no required status checks from branch protection or rulesets, so this does not block.
  • I checked save_pack with --etag against Hive.set. It sends data and no usr_mtd, so the server keeps the stored metadata and a replace does not disable the pack. The create path sends enabled: true. Neither path needs a change.

Blast Radius

  • New SDK methods Investigation.get_pack, save_pack and delete_pack, plus the new CLI group limacharlie investigate pack (get, save, delete). All of these are additive.
  • investigate packs --brief now passes through version, digest, source, trust and trust_provenance when the server returns them. Strict consumers of the JSON, CSV or table output will see new fields or columns.
  • The investigation_engine discovery profile now lists three more commands.

This review was generated automatically by Claude Code.

@maximelb

Copy link
Copy Markdown
Contributor Author

/lc-review

@limacharlie-event-notifications limacharlie-event-notifications Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI Code Review Summary

No high or medium issues at this commit. The newest commit changes only docs and tests. It adds resolved_packs notes and two tests that check validate and test responses are passed through unchanged. One low-priority docs note is inline.

General Feedback

  • test-time-budget: no check run was found on this commit. The base branch w10/explicit-investigation-trust has no required status checks in branch protection or rulesets, so this does not block.
  • The new tests (test_validate_and_initial_test_preserve_each_resolved_pack and test_validation_and_test_json_preserve_resolved_packs) only check pass-through. validate_plan, test_plan and _compact_test do not read resolved_packs, children or trust_provenance. These tests guard against future filtering on the client side. They do not cover resolution logic, which runs on the server. This is acceptable, but keep that limit in mind when you describe what the tests cover.

Blast Radius

  • New SDK methods Investigation.get_pack, save_pack and delete_pack, plus the new CLI group limacharlie investigate pack (get, save, delete). All of these are additive.
  • investigate packs --brief now passes through version, digest, source, trust and trust_provenance when the server returns them. Strict consumers of the JSON, CSV or table output will see new fields or columns.
  • The investigation_engine discovery profile now lists three more commands.

This review was generated automatically by Claude Code.

Comment thread doc/cli/investigation-engine.md Resolved
@maximelb

Copy link
Copy Markdown
Contributor Author

/lc-review

@limacharlie-event-notifications limacharlie-event-notifications Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI Code Review Summary

No issues found at this commit. The newest commit changes only doc/cli/investigation-engine.md. It adds a use fact example with pack_version and pack_digest, and says how pack_digest differs from snapshot_digest. This fixes the earlier docs note. The SDK, CLI and test files have not changed since the last review, and that review found no high or medium issues in them.

General Feedback

  • test-time-budget: no check run was found on this commit. Branch protection and rulesets for w10/explicit-investigation-trust list no required checks, so this does not block.
  • Docs nit: the new ## Org-authored fact packs... section still comes after ## See Also. Readers usually expect See Also to be the last section. This is optional.

Blast Radius

  • New SDK methods Investigation.get_pack, save_pack and delete_pack, plus the new CLI group limacharlie investigate pack (get, save, delete). All of these are additive.
  • investigate packs --brief now passes through version, digest, source, trust and trust_provenance when the server returns them. Strict consumers of the JSON, CSV or table output will see new fields or columns.
  • The investigation_engine discovery profile now lists three more commands.

This review was generated automatically by Claude Code.

Base automatically changed from w10/explicit-investigation-trust to master November 14, 2026 23:37
@maximelb
maximelb dismissed lcbill’s stale review November 14, 2026 23:37

The base branch was changed.

@maximelb

Copy link
Copy Markdown
Contributor Author

/lc-review

@limacharlie-event-notifications limacharlie-event-notifications Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI Code Review Summary

No issues found at this commit. The PR now targets master. I re-read every changed file. The SDK, CLI and test changes match the earlier reviewed versions. One addition: save_pack now rejects a non-string name, and the tests cover it. The docs now put the composition section and the --follow note above ## See Also, which fixes the earlier docs nit.

General Feedback

  • test-time-budget: no check run was found on this commit. Neither branch protection nor rulesets for master list any required checks, so this does not block.
  • The composition and resolved_packs tests only check that data passes through unchanged. Composition and digest enforcement run on the server, so these tests cannot catch regressions there. The PR body already says this.

Blast Radius

  • New public SDK methods: Investigation.get_pack, save_pack and delete_pack. New CLI group: limacharlie investigate pack (get, save, delete). All are additive.
  • investigate packs --brief now includes version, digest, source, trust and trust_provenance when the server returns them. Strict consumers of the JSON, CSV or table output will see new fields or columns.
  • The investigation_engine discovery profile lists three more commands, and the investigate lazy-loading subcommand set now includes pack.

This review was generated automatically by Claude Code.

@maximelb

Copy link
Copy Markdown
Contributor Author

Maintainer review (adversarial): new SDK get_pack / save_pack / delete_pack on the investigation_pack hive, plus investigate pack get|save|delete. save_pack validates the name, non-empty YAML and the etag, and defaults to create-only. Delete requires --confirm. Record names are URL-escaped with safe='' by the Hive client, so a name can't escape its path. The composition docs match behaviour, and the server enforces composition and digests (as the body says). No private references in the diff or commits. No issues found.

@maximelb
maximelb merged commit c21d316 into master Nov 14, 2026
7 checks passed
@maximelb
maximelb deleted the feat/w11-surfaces branch November 14, 2026 23:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants