Skip to content

stack-encrypt refactor: complete configured definitions and typed query APIs #1486

Closed

Description

@coderdan

Background

Stack Encrypt (packages/stack-encrypt) is the Rust engine that every CipherStash language SDK runs on. A Plan is a record's declaration as data: its name scope, its fields, and each field's indexes. It says how each field is stored, and which queries each field supports. Writes and queries must use the same plan. Otherwise a query derives terms that do not match what was stored, and finds nothing.

Stack Encrypt has two layers. The typed Rust API is checked by the compiler. The dynamic interface is behind the dynamic feature. It is for language bindings such as the Go SDK. Their plans and values arrive as data.

The Stack Encrypt API plan, merged in pull request #1365, decided how both layers declare and query a record.

Problem

On next, a Rust program writes a query plan by hand: cipher.query(op, value).on(&plan). Nothing ties that plan to the plan the record was written with. Some operators, such as element match, exist only in the dynamic interface. Each binding also builds its own requests, so two bindings can come to disagree.

Proposal

The plan items "Field markers and the encryption plan" and "Planned and its serialised form" describe the work.

  1. Field markers. The derive emits one marker per field, as an associated constant such as EncryptedMember::EMAIL. The marker has the field's name, plaintext type, accessor, and index list. It offers one method per operator the field's indexes answer, checked at compile time.
  2. encryption_plan(). The derive emits it. It returns &'static Plan, built once through the plan builder.
  3. A hand-built plan gets the same markers. Plan::<Member>::scope("members").fields() followed by the field verbs. build() returns the plan and one marker per field.
  4. One validator for options. Typed Rust option factories and options read from data go through the same validator, before any key is fetched.
  5. Two wire shapes for bindings. A serialised Plan alone serves a check, an operator list, or a query plan. A serialised Planned serves encrypt, query, and decrypt. It contains the plan, the context parts in declared order, and the value. It is a transport form only. It is never logged or stored, because its value is plaintext.
  6. Query plans. plan.query(field, op) is refused where it is made if no index answers the operator.
  7. The dynamic field handle. plan.field(name), with every operator checked at run time.
  8. contexts() on a Plan and on a Planned, before either runs.

Typed JSON queries are #1459. Shared query preparation and refusals are #1279 and its child issues.

Acceptance

  • The plan's marker example compiles.
  • tests/ui compile-fail cases cover an operator the field's indexes do not answer.
  • Tests show that a query cannot run without its plan or its required context.
  • Every case in the conformance corpus is a serialised Plan or Planned, with the engine's expected answer beside it.
  • A Go program generated for rc.3 runs against a module that sends serialised Planned values. The only change allowed is the one regeneration the handle rename in stack-encrypt refactor: finalize crate ownership, naming and coordinated migration #1489 needs.
  • Custom Rust stages that are not registered stay explicitly non-portable, and tests cover the refusal.
  • Do not reimplement the configured JSON target support that Go already has.

Relationship to other work

Plans: the Stack Encrypt API plan and the conformance roadmap.

Activity

  1. self-assigned this
    on Nov 1, 2026
  2. coderdan commented on Nov 9, 2026

    @coderdan
    ContributorAuthor

    For the API plan's "EQL types as fields" item, which this issue covers in part: its prerequisite is now tracked in #1585. That issue adds EncryptTarget and DecryptSource to the EQL v4 types on their own. Fields of a derived record also need the record's context handed to an EQL field as an Identifier, which #1585 leaves out.

    #1586 is related: it lets a target take its context from the plaintext, which the EQL migration and compatibility shims use.

  3. coderdan commented on Nov 9, 2026

    @coderdan
    ContributorAuthor

    The field-marker form of assert_context! from #1568 needs the field markers this issue adds. #1590 tracks it.

  4. coderdan commented on Nov 9, 2026

    @coderdan
    ContributorAuthor

    Pull request #1595, stacked on #1592, does items 1, 6, 7, and 8 of this issue. Those are the field markers, query plans, the dynamic field handle, and contexts() on a Plan and a Planned. It also adds Planned itself.

    Item 5, the two wire shapes for bindings, is split into #1594. The two acceptance items about the corpus and the Go module go with it.

    Items 2 and 3 are encryption_plan(), and markers on a hand-built plan. The plan's decision of 7 November 2026 withdrew both. The derive emits no plan. No Rust program builds one by hand.

    Item 4, one validator for options, landed with Supports::prepare in #1592.

  5. added 2 commits that reference this issue on Nov 9, 2026
    99c1cca
    9a58106
  6. coderdan commented on Nov 9, 2026

    @coderdan
    ContributorAuthor

    Two acceptance items, from the review of #1595: the marker example in the plan compiles as written now, with the forms #1595 built (.between(&18, &65) and a ValueFrame under a query plan). The custom-stages item goes to #1594 with the Go module and the corpus. The operator methods a marker still lacks are #1596.

  7. added 6 commits that reference this issue on Nov 9, 2026
    7c11632
    38894b0
    61253fb
    8290a58
    04e5c4a
    409c918
  8. 16 remaining items

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions