The test corpus

90 cases. A conformance claim is proven by passing a level's folders; the corpus is part of the spec.

A conformance claim (spec README, "Conformance levels") is proven by passing that level's folders here. The corpus is part of the spec: when a case and a chapter disagree, one of them is a bug and gets fixed.

Layout #

corpus/
  reader/            cases every Reader must pass
    _package.crubal  the kinds and levels the cases use
    _facets.crubal   the facet definitions the cases use (shorthand facet lines)
    001-scalars/
      input.crubal   text in
      expected.json  the data it must read to
  reader-errors/     input a Reader must refuse
    e01-tab-indent/
      input.crubal
      error.txt      line number, then a short description of the error
  dictionary/        cases every Dictionary-level implementation must pass
    _facets.crubal   the definitions the data cases use
    _classes.crubal  the classes the class cases use
    001-…/
      input.crubal   definitions or data in
      long.crubal    (definition cases) the same definitions in the long form
      expected.json  the definitions (JSON, chapter 4.6) or the folded data it must read to
      expected-outcomes.json  (check cases) the outcome for each address (chapters 4.9, 5.5)
    a `resolve: Class` input expects the resolved class lines (chapter 5.4)
  dictionary-errors/ definitions a package publish must refuse
  rules/             cases every Rules-level implementation must pass
    _facets.crubal · _rules.crubal   the facets and rules every case runs under
    001-…/
      input.crubal   records written, in order
      write.crubal   (optional) a later write to those records, read like input
      expected-results.json  the rule results after the last write (chapter 6.9)
  rules-errors/      rule sets a package publish must refuse

A rules/ case MAY hold a now.txt: the moment now means in its conditions (chapter 6.3).

rules/ results are compared by kind: for every result kind a case lists (complete, filled, derived, flagged, incomplete, refused, held, completeness), the set of (record, address) pairs of that kind must match exactly, and each listed field must be equal. Kinds a case doesn't list aren't compared.

store/ cases hold a log.json and expected-verify.json (does the chain check out, and where does it break), or a record, an intake.json and the expected.json record after it lands. Intake cases add a contract.crubal, an answer.json and expected-intake.json; evidence cases add observations.json and expected-validation.json. Signed-chain cases add a keys.json (the owner's published public key, from store/_keys.json); a chain case without one checks links only. Replay cases add contract-v2.crubal and expected-replay.json; review cases add intakes.json and expected-review.json; raw-byte cases add answer.bin and expected-raw.json. tools/ holds the generators for cases that need real hashes or signatures. Migration cases add a recipe.crubal and expect the restamped data plus its review queue; store-errors/ holds recipes a publish must refuse. package-errors/ holds packages a publish must refuse (chapter 11.5). views/ cases project a view over _records.crubal and expect expected-projection.json or, with a pick:, expected-counts.json. exchange/ cases add run.crubal (which contract runs on which record), an optional answer.json (what the other side sent back) and expected-run.json. A run.crubal may hold several runs in order, each run: with on:, and optionally caller: and offer:; write: ID with indented values changes a record between runs. The runner names each run's log entry L-R1, L-R2 …, in order. Offer-and-take cases add the presenter's record as presenter.json; border cases hold a signed wrapper in border.json, the presenter's key in keys.json and expected-border.json.

Passing a case #

For a case with expected.json, a Reader passes when all three hold:

1. reading input.crubal gives JSON whose canonical form (chapter 3.9) equals the canonical form of expected.json;
2. printing that JSON in full mode and reading the print gives the same canonical form;
3. reading expected.json and printing it gives text that reads to the same canonical form.

Layout, quoting and comment choices never decide a pass: only the pairs are compared.

For a case with error.txt, a Reader passes when it refuses the input and names the given line. The wording of the message is free.

A case's input.crubal MAY start with definitions (facets, classes, rules) that apply to that case only; the data part begins at the crubal: header line.

Writing a case #

• One idea per case, named for it.
• Use the shared _package.crubal and _facets.crubal where they fit. Definitions only one case needs go at the top of that case's input.
• The owner is test.example and the package is test, so the corpus stays independent of any adopter.