parent:
f06eb39c982d1ea91f18e6b8af90fc31f87ae5f0
nmyk <nick@nmyk.io>
2026-11-14T14:52:06-05:00
design and plan updates remove explicit red tape pick special root OID
diff --git a/DESIGN.md b/DESIGN.md
index 70cf23920b10b815ee23aa54683975a5bf91e8ea..fb3feb506650a010b332f89dc85f699e8c32dd58 100644
--- a/DESIGN.md
+++ b/DESIGN.md
@@ -22,10 +22,10 @@
1. **Load:** decode the PEM blocks into certificates.
2. **Ratify:** validate the path (name chaining, signatures, validity, CA constraints).
3. **Policy processing:** build the valid policy tree per RFC 5280 §6.1.3–6.1.4 as updated by RFC 9618.
-4. **Decode:** turn the tree into an AST (drop padding, order children, decode OIDs into atoms).
+4. **Decode:** turn the tree into an AST (skip the validator's copies, order children, decode OIDs into atoms).
5. **Evaluate:** evaluate each top-level form in order.
-Terminology used in this doc: *GOV* is the language's root OID arc; *red tape* is the padding node that keeps every leaf at the same depth; *ratify* means "pass path validation".
+Terminology used in this doc: *GOV* is the language's root OID arc; *red tape* is the copies of a leaf that the validator itself generates, through `anyPolicy`, to keep every leaf at the same depth; *ratify* means "pass path validation".
## Source format
@@ -51,7 +51,7 @@ | subjectAltName | author email, if given | absent | absent |
| Validity | `-effective` to `-expires` (default 2000-01-01 to 9999-12-31) | same | same |
| basicConstraints (critical) | CA:TRUE | CA:TRUE | CA:FALSE |
| keyUsage (critical) | keyCertSign, cRLSign | keyCertSign, cRLSign | digitalSignature |
-| certificatePolicies | absent | node OIDs at depth i, each with optional notes | node OIDs at depth n, each with optional notes |
+| certificatePolicies | absent | cert 1: the top-level node OIDs; certs 2..n−1: `anyPolicy` only, plus any node that carries notes | `anyPolicy` only, plus any node that carries notes |
| policyMappings (critical) | absent | one entry per parent→child edge | must be absent |
| SKI / AKI | present | present | present |
@@ -62,7 +62,9 @@ - Ed25519 signatures are deterministic, so fixed keys plus fixed dates give byte-identical output.
- Mappings in cert *n* are rejected because RFC 5280 never processes them (§6.1.4 runs only for certs 1..n−1); allowing them would hide transpiler bugs.
- Any unrecognized critical extension is rejected, per RFC 5280.
-**Size.** A locally built prototype cert with 1 policy and 3 mappings was 525 bytes DER. Expect roughly 350 bytes per depth plus about 70–120 bytes per node (its policy entry plus its incoming mapping), before PEM's 4/3 base64 overhead.
+**Size.** A locally built prototype cert with 1 policy and 3 mappings was 525 bytes DER. Only cert 1 lists nodes; every other node costs just its incoming mapping, so expect roughly 350 bytes per depth plus about 50 bytes per node, before PEM's 4/3 base64 overhead.
+
+A program whose forms are all atoms has depth 1: its single cert is both cert 1 and cert n, and lists the top-level nodes.
## The policy tree as AST
@@ -74,46 +76,47 @@ RFC 5280 §6.1 builds the tree one certificate at a time:
1. **Init:** the root is a single `anyPolicy` (`2.5.29.32.0`) node at depth 0, expecting `{anyPolicy}`.
2. **Policies of cert i:** for each policy P in the cert, every depth *i−1* node whose `expected_policy_set` contains P gets a child P. If none matches, P attaches to a depth *i−1* `anyPolicy` node, which exists only at depth 0. So cert 1's policies become the root's children: the top-level forms.
-3. **Prune:** any node at depth *i−1* or less with no children is deleted, repeatedly.
-4. **Mappings of cert i:** for every depth-*i* node whose policy is an `issuerDomainPolicy`, its `expected_policy_set` becomes the set of mapped `subjectDomainPolicy` values. An unmapped node expects only itself.
+3. **anyPolicy in cert i** (RFC 9618 step d.2): for every policy that some depth-*i−1* node expects but that has no node at depth *i* yet, create that node, with every node that expects it as a parent. Certs 2..n assert only `anyPolicy`, so this one rule creates every node below depth 1: a mapped node's children, and, for a node with no mappings, a copy of itself.
+4. **Prune:** any node at depth *i−1* or less with no children is deleted, repeatedly.
+5. **Mappings of cert i:** for every depth-*i* node whose policy is an `issuerDomainPolicy`, its `expected_policy_set` becomes the set of mapped `subjectDomainPolicy` values. An unmapped node expects only itself, which is why step 3 copies it.
RFC 9618 (2024) updates this: it replaces the tree with a `valid_policy_graph`, a DAG where a node may have several parents, to avoid exponential growth. When every node has one parent, the graph and the tree are the same structure.
-**Validation inputs:** `user-initial-policy-set = {anyPolicy}`, `initial-policy-mapping-inhibit = false`, `initial-explicit-policy = true`, `initial-any-policy-inhibit = true`. Explicit policy makes an empty tree a validation failure.
+**Validation inputs:** `user-initial-policy-set = {anyPolicy}`, `initial-policy-mapping-inhibit = false`, `initial-explicit-policy = true`, `initial-any-policy-inhibit = false`. Explicit policy makes an empty tree a validation failure. `anyPolicy` must not be inhibited, because certs 2..n depend on it; for the same reason the `inhibitAnyPolicy` extension is rejected.
### Constraints and design responses
| # | Constraint from RFC 5280 / 9618 | Consequence for a Lisp AST | GOVERNMENT's response |
| --- | --- | --- | --- |
-| 1 | Pruning deletes every leaf above depth n | `(+ 1 (* 2 3))` would lose `+` and `1` | **Red tape**: a chain of padding nodes under each shallow atom, down to depth n |
+| 1 | Pruning deletes every leaf above depth n | `(+ 1 (* 2 3))` would lose `+` and `1` | **Red tape**: certs 2..n assert `anyPolicy`, so the validator copies every unmapped leaf down to depth n. Nothing is written for it |
| 2 | A cert's policies are a set (no duplicates, no order) | `(+ 1 1)` cannot have two `1` children; argument order is lost | A **node-id arc** in every OID makes siblings distinct and gives their order |
| 3 | Mappings key on the OID, not the node | Two equal-OID nodes at one depth would get identical children | Node ids are **unique across the whole program**, so every OID is unique |
-| 4 | An unmapped node expects only its own OID | With unique ids, an interior node that lacks mappings loses its subtree | **Strict check**: every interior node must be mapped (see below) |
-| 5 | A policy matching no parent at depth ≥ 2 is silently dropped | A typo in a mapping deletes code without an error | **Strict check**: every asserted policy must appear in the final tree |
+| 4 | An unmapped node expects only its own OID | A missing mapping silently drops a subtree: nodes below depth 1 exist only through mappings | A list node (code 1) must have at least one element, and the empty list is its own constant (code 5). **Strict check**: node ids must be exactly 1..N, so a lost subtree leaves a gap. Only losing the subtree with the highest ids goes unnoticed; signatures rule out tampering, so that case can only be a transpiler bug, which round-trip tests catch |
+| 5 | A policy matching no parent at depth ≥ 2 is silently dropped | An explicitly listed node (one carrying notes) with a wrong OID would vanish | **Strict check**: every explicitly asserted policy must appear in the final tree |
| 6 | Mappings in cert n are never processed | Dead data | Rejected |
-| 7 | `anyPolicy` must not be mapped | — | `anyPolicy` may not appear in any cert |
+| 7 | `anyPolicy` must not be mapped | — | Certs 2..n must assert `anyPolicy`; cert 1 must not (it would create an `anyPolicy` node under the root); it may never be mapped |
| 8 | An empty tree fails explicit-policy validation | The empty program cannot exist | A program needs ≥ 1 top-level form |
| 9 | One child OID mapped from several parents gets several parents (9618) or is duplicated (5280) | Shared subexpressions | Allowed; evaluated directly on the DAG, never unfolded (decided) |
-**Strict check.** Plain RFC processing forgives mistakes by deleting code. After policy processing, the interpreter also verifies that every policy asserted in cert *i* is a node at depth *i*, and that every mapping in cert *i* produced an edge. A failure is a ratification error naming the orphaned OID. This turns silent data loss into a compile-style error.
+**Strict check.** Plain RFC processing forgives mistakes by deleting code. After policy processing, the interpreter also verifies that certs 2..n assert `anyPolicy` and cert 1 does not, that every explicitly asserted policy in cert *i* is a node at depth *i*, that every mapping in cert *i* produced an edge, and that the program's distinct node ids are exactly 1..N with no gaps. A failure is a ratification error naming the orphaned OID. This turns silent data loss into a compile-style error.
**Denial-of-service guard.** Policy processing has caused real DoS bugs, e.g. Go's CVE-2026-32281 (inefficient policy validation, fixed in Go 1.25.9 / 1.26.2). The graph form keeps work linear in mappings, and the interpreter caps nodes (`-max-nodes`, default 1,000,000).
## OID encoding
-Every node OID is `GOV.<namespace>.<node-id>.<payload…>`, where the payload is the Unicode code points of the node's text, one arc each (so ASCII `A` is arc 65).
+Every node OID is `GOV.<namespace>.<node-id>.<payload…>`. For a builtin the payload is one arc, its fixed code from the builtin table. For every other node it is the Unicode code points of the node's text, one arc each (so ASCII `A` is arc 65).
-**Root arc.** `GOV = 2.25.54990973348924144048682522826038955666`. The `2.25` arc takes a UUID as an integer and needs no registration (ITU-T X.667); this one is UUID `295ee050-3d82-4b0b-9b04-adbf3de5ce92`. Go's `x509.ParseOID` handles the 128-bit arc (verified locally). Tools print `GOV` as shorthand.
+**Root arc.** `GOV = 2.25.9665536045854453410170579537759926404`. The `2.25` arc takes a UUID as an integer and needs no registration (ITU-T X.667); this one is UUID `074583b8-3bb7-4179-8669-827877697884`. Go's `x509.ParseOID` handles the 128-bit arc. Tools print `GOV` as shorthand.
**Node id.** A positive integer, unique across the whole program, assigned in pre-order by the transpiler. Siblings are ordered by ascending id. Uniqueness is what makes constraints 2 and 3 above go away.
-**Payload.** Unicode code points 0–0x10FFFF, excluding surrogates. ASCII is the common case. Verified locally: an arc of 128512 (😀) survives a create/parse round trip.
+**Payload.** For symbols, numbers and strings: Unicode code points 0–0x10FFFF, excluding surrogates. ASCII is the common case. Verified locally: an arc of 128512 (😀) survives a create/parse round trip.
### Namespaces
| ns | Kind | Payload | Source | OID |
| --- | --- | --- | --- | --- |
-| 0 | builtin | builtin name; empty = red tape | `+` | `GOV.0.7.43` |
+| 0 | builtin | one arc, the builtin's code | `+` | `GOV.0.7.21` |
| 1 | symbol | symbol name (non-empty) | `foo` | `GOV.1.8.102.111.111` |
| 2 | integer | decimal text | `-42` | `GOV.2.9.45.52.50` |
| 3 | float | decimal text | `3.14` | `GOV.3.10.51.46.49.52` |
@@ -131,74 +134,80 @@ **Why text for numbers.** It is uniform with strings and symbols, follows your ASCII-arc idea, round-trips floats exactly as written, and is readable in an `openssl` dump with a one-line decoder. The alternative (a sign arc plus a magnitude arc for integers, IEEE-754 bits for floats) is shorter but opaque.
## Builtin symbol table
-There are 28 builtins, all in namespace 0, each spelled as the ASCII codes of its name; `<id>` is the node's own id. Names are case-sensitive.
+There are 28 builtins, all in namespace 0. Each has a fixed numeric code, which is its whole payload: one arc. `<id>` is the node's own id. Names are case-sensitive.
-| Name | Kind | OID | Arity / note |
-| --- | --- | --- | --- |
-| `()` | structural | `GOV.0.<id>.40.41` | List node; with no elements it is the empty list |
-| `.` | structural | `GOV.0.<id>.46` | Dotted-pair marker inside a quoted list |
-| (empty) | structural | `GOV.0.<id>` | Red tape (padding) |
-| `T` | constant | `GOV.0.<id>.84` | Boolean true |
-| `F` | constant | `GOV.0.<id>.70` | Boolean false |
-| `quote` | special form | `GOV.0.<id>.113.117.111.116.101` | 1 operand |
-| `if` | special form | `GOV.0.<id>.105.102` | 2 or 3 operands |
-| `define` | special form | `GOV.0.<id>.100.101.102.105.110.101` | 2 operands |
-| `lambda` | special form | `GOV.0.<id>.108.97.109.98.100.97` | params + 1 or more body forms |
-| `and` | short-circuit form | `GOV.0.<id>.97.110.100` | 0+ operands |
-| `or` | short-circuit form | `GOV.0.<id>.111.114` | 0+ operands |
-| `not` | function | `GOV.0.<id>.110.111.116` | 1 arg |
-| `+` | function | `GOV.0.<id>.43` | 0+ args |
-| `-` | function | `GOV.0.<id>.45` | 1+ args |
-| `*` | function | `GOV.0.<id>.42` | 0+ args |
-| `/` | function | `GOV.0.<id>.47` | 1+ args |
-| `<` | function | `GOV.0.<id>.60` | 2 args |
-| `>` | function | `GOV.0.<id>.62` | 2 args |
-| `<=` | function | `GOV.0.<id>.60.61` | 2 args |
-| `>=` | function | `GOV.0.<id>.62.61` | 2 args |
-| `==` | function | `GOV.0.<id>.61.61` | 2 args |
-| `!=` | function | `GOV.0.<id>.33.61` | 2 args |
-| `cons` | function | `GOV.0.<id>.99.111.110.115` | 2 args |
-| `car` | function | `GOV.0.<id>.99.97.114` | 1 arg |
-| `cdr` | function | `GOV.0.<id>.99.100.114` | 1 arg |
-| `strlen` | function | `GOV.0.<id>.115.116.114.108.101.110` | 1 arg |
-| `strcat` | function | `GOV.0.<id>.115.116.114.99.97.116` | 0+ args |
-| `str?` | function | `GOV.0.<id>.115.116.114.63` | 1 arg |
+Codes are grouped with room to grow: structure 1–2, constants 3–5, special forms 10–15, functions 20 and up. A code is permanent: it is never reused or renumbered, and a new builtin takes the next free code in its group.
-A ns-0 OID whose name is not in this table is a decode error. Adding a builtin later is backward compatible: it is a new name in the same namespace.
+| Code | Name | Kind | OID | Arity / note |
+| --- | --- | --- | --- | --- |
+| 1 | `(…)` | structural | `GOV.0.<id>.1` | List node; its children are the elements; at least one |
+| 2 | `.` | structural | `GOV.0.<id>.2` | Dotted-pair marker inside a quoted list |
+| 3 | `T` | constant | `GOV.0.<id>.3` | Boolean true |
+| 4 | `F` | constant | `GOV.0.<id>.4` | Boolean false |
+| 5 | `()` | constant | `GOV.0.<id>.5` | The empty list |
+| 10 | `quote` | special form | `GOV.0.<id>.10` | 1 operand |
+| 11 | `if` | special form | `GOV.0.<id>.11` | 2 or 3 operands |
+| 12 | `define` | special form | `GOV.0.<id>.12` | 2 operands |
+| 13 | `lambda` | special form | `GOV.0.<id>.13` | params + 1 or more body forms |
+| 14 | `and` | short-circuit form | `GOV.0.<id>.14` | 0+ operands |
+| 15 | `or` | short-circuit form | `GOV.0.<id>.15` | 0+ operands |
+| 20 | `not` | function | `GOV.0.<id>.20` | 1 arg |
+| 21 | `+` | function | `GOV.0.<id>.21` | 0+ args |
+| 22 | `-` | function | `GOV.0.<id>.22` | 1+ args |
+| 23 | `*` | function | `GOV.0.<id>.23` | 0+ args |
+| 24 | `/` | function | `GOV.0.<id>.24` | 1+ args |
+| 25 | `<` | function | `GOV.0.<id>.25` | 2 args |
+| 26 | `>` | function | `GOV.0.<id>.26` | 2 args |
+| 27 | `<=` | function | `GOV.0.<id>.27` | 2 args |
+| 28 | `>=` | function | `GOV.0.<id>.28` | 2 args |
+| 29 | `==` | function | `GOV.0.<id>.29` | 2 args |
+| 30 | `!=` | function | `GOV.0.<id>.30` | 2 args |
+| 31 | `cons` | function | `GOV.0.<id>.31` | 2 args |
+| 32 | `car` | function | `GOV.0.<id>.32` | 1 arg |
+| 33 | `cdr` | function | `GOV.0.<id>.33` | 1 arg |
+| 34 | `strlen` | function | `GOV.0.<id>.34` | 1 arg |
+| 35 | `strcat` | function | `GOV.0.<id>.35` | 0+ args |
+| 36 | `str?` | function | `GOV.0.<id>.36` | 1 arg |
-`()` uses your suggested 40/41: rather than appearing as separate open and close tokens, the pair names the list node itself, and the node's children are the list's elements. Quote sugar (`'x`) has no node; the transpiler expands it to `(quote x)`.
+A ns-0 OID whose code is not in this table, or that does not have exactly one payload arc, is a decode error. Adding a builtin later is backward compatible: it is a new code in the same namespace.
+
+Fixed codes keep every builtin OID to one payload arc (spelling names in ASCII took up to six), at the cost of `openssl` dumps no longer showing builtin names; `government tree` decodes them.
+
+A non-empty list is one list node whose children, in id order, are the list's elements, rather than separate open and close tokens. The tree already encodes nesting, so this needs no matching of parentheses, makes unbalanced lists impossible to encode, and keeps depth equal to nesting depth. Quote sugar (`'x`) has no node; the transpiler expands it to `(quote x)`.
## Tree to AST decoding
-Decoding strips red tape, sorts each node's children by id, and reads every OID as an atom or a list.
+Decoding skips red tape (the validator's copies), sorts each node's children by id, and reads every OID as an atom or a list.
**Rules**
1. The root (`anyPolicy`) is the program. Its children, in id order, are the top-level forms.
-2. A `()` node is a list. Its children, in id order, are its elements. A `()` whose only child is red tape (or that sits at depth n) is the empty list.
-3. Every other node is an atom. It must be at depth n, or have exactly one child that is red tape.
-4. A red-tape node has zero or one child, and that child must be red tape. Red tape may appear only under an atom, under an empty `()`, or under red tape.
+2. A child with the same OID as its parent is a copy made by the validator. The decoder skips it. A copy may only have a copy below it.
+3. A list node (code 1) is a list. Its other children, in id order, are its elements, and it must have at least one. The empty list is the constant `()` (code 5), an atom.
+4. Every other node is an atom. Its only child may be its copy.
5. `.` may appear only as the second-to-last element of a list. It builds a dotted pair and is only meaningful under `quote`.
6. Anything else is a decode error that names the offending OID and depth.
### Worked example: `(+ 1 (* 2 3))`
-The AST is 3 deep, so the file holds an anchor plus 3 certs. Pre-order ids, with red tape (`∅`) padding `+` and `1` down to depth 3:
+The AST is 3 deep, so the file holds an anchor plus 3 certs. Pre-order ids; `′` marks a copy the validator makes:
```
depth 0 anyPolicy
-depth 1 └─ ()#1
-depth 2 ├─ +#2 ├─ 1#4 └─ ()#6
-depth 3 │ └─ ∅#3 │ └─ ∅#5 ├─ *#7 ├─ 2#8 └─ 3#9
+depth 1 └─ (…)#1
+depth 2 ├─ +#2 ├─ 1#3 └─ (…)#4
+depth 3 │ └─ +#2′ │ └─ 1#3′ ├─ *#5 ├─ 2#6 └─ 3#7
```
| Cert | certificatePolicies | policyMappings |
| --- | --- | --- |
-| 1 | `GOV.0.1.40.41` | `GOV.0.1.40.41 → GOV.0.2.43`, `→ GOV.2.4.49`, `→ GOV.0.6.40.41` |
-| 2 | `GOV.0.2.43`, `GOV.2.4.49`, `GOV.0.6.40.41` | `GOV.0.2.43 → GOV.0.3`; `GOV.2.4.49 → GOV.0.5`; `GOV.0.6.40.41 → GOV.0.7.42`, `→ GOV.2.8.50`, `→ GOV.2.9.51` |
-| 3 | `GOV.0.3`, `GOV.0.5`, `GOV.0.7.42`, `GOV.2.8.50`, `GOV.2.9.51` | none |
+| 1 | `GOV.0.1.1` | `GOV.0.1.1 → GOV.0.2.21`, `→ GOV.2.3.49`, `→ GOV.0.4.1` |
+| 2 | `anyPolicy` | `GOV.0.4.1 → GOV.0.5.23`, `→ GOV.2.6.50`, `→ GOV.2.7.51` |
+| 3 | `anyPolicy` | none |
-Policy processing reproduces the tree above. Cert 1's policy attaches to `anyPolicy`. Each later policy attaches to the one parent whose mapping names it. Nothing is pruned, because every branch reaches depth 3. Decoding yields `(+ 1 (* 2 3))`, which evaluates to `7`.
+Policy processing reproduces the tree above. Cert 1's policy attaches to the root. In cert 2, `anyPolicy` creates the three nodes #1 expects. In cert 3, it creates the three nodes #4 expects, and copies `+#2` and `1#3`, which expect only themselves. Nothing is pruned, because every branch reaches depth 3. Decoding skips the copies and yields `(+ 1 (* 2 3))`, which evaluates to `7`.
+
+This was checked with a prototype chain against two independent implementations, OpenSSL 3.0 and Go 1.24's `x509.Verify`: a shallow atom survives to the last cert only when the certs below it assert `anyPolicy`, mapping targets need not be listed, and inhibiting `anyPolicy` makes the chain fail.
## Language semantics
@@ -279,7 +288,7 @@ ### Go facts verified locally (go1.24.7)
- `x509.OID` and `x509.ParseOID` handle arcs beyond 64 bits, including the 128-bit `GOV` arc.
- `Certificate.Policies` and `Certificate.PolicyMappings` are populated by `ParseCertificate`, and a critical policyMappings extension is not listed as unhandled.
-- **`CreateCertificate` does not emit `PolicyMappings` in 1.24.** It silently drops the field. The transpiler must DER-encode the extension itself and pass it through `ExtraExtensions`. A ~15-line hand encoder was enough in the prototype.
+- **`CreateCertificate` does not emit `PolicyMappings` in 1.24.** It silently drops the field. The transpiler must DER-encode the extension itself and pass it through `ExtraExtensions`. A ~15-line hand encoder was enough in the prototype. The same is true of `policyConstraints`.
- Ed25519 certificate signing is byte-for-byte deterministic.
**Why not `x509.Verify`?** It builds its own paths (we need the exact file order), it does not expose the policy tree, and it is built around system-root semantics. `CheckSignatureFrom` gives us signatures plus the parent's CA and keyCertSign checks.
@@ -291,7 +300,7 @@ | --- | --- |
| `cmd/government` | CLI |
| `ratify` | PEM loading; path validation (§6.1.3 a–c, 6.1.4 k–n); critical-extension check |
| `policy` | RFC 9618 policy graph; strict check; node budget |
-| `ast` | Graph to AST: red tape, ordering, OID decoding, structural errors |
+| `ast` | Graph to AST: skipping copies, ordering, OID decoding, structural errors |
| `eval` | Values, environments, special forms, builtins, trampoline for tail calls |
| `sexpr` | Lisp reader and printer, shared by the transpiler, decompiler and tests |
| `lisp2gov` | Transpiler: s-expressions to `.gov` |
@@ -330,8 +339,8 @@
1. **Read** Lisp: integers, floats, strings (escapes `\"` `\\` `\n` `\t` `\u{…}`), symbols, `T`/`F`, `'x`, `;` comments (kept as notes), dotted pairs.
2. **Classify**: reserved names go to ns 0, other symbols to ns 1, literals to ns 2–4. Expand `'x` to `(quote x)`. Attach each comment to its form.
3. **Number** nodes in pre-order, starting at 1.
-4. **Pad**: compute n = max depth; hang a red-tape chain under every atom and empty list shallower than n.
-5. **Group** nodes by depth into per-cert policy sets with their notes; turn each parent→child edge into a mapping in the parent's cert.
+4. **Layer**: compute n = max depth and each node's depth. No padding is written: `anyPolicy` makes the validator copy shallow leaves down.
+5. **Assign**: cert 1 lists the top-level nodes; certs 2..n list `anyPolicy`, plus any node that carries notes; each parent→child edge becomes a mapping in the parent's cert.
6. **Sign**: take the author key (or the default seed), derive one Ed25519 key per depth, build the anchor and certs 1..n, and hand-encode certificatePolicies and policyMappings.
7. **Emit** PEM, with the original source as a header comment above the first block.
@@ -372,10 +381,10 @@ | --- | --- | --- |
| Language | `testdata/*.lisp` | All semantics: arithmetic, comparisons, truthiness, `if`, `quote`, lists, `define`, closures, scoping, recursion, higher-order functions, strings and Unicode |
| Shape stress | `testdata/*.lisp` | 200-deep nesting (201 certs), 1,000-argument calls, wide `quote`d lists, shared subexpressions |
| Round trip | Go test over every `testdata` file | `gov2lisp(lisp2gov(src))` equals the normalized source, comments included |
-| Ratification | Go table test that mutates a valid chain | Flipped signature byte, reordered certs, dropped mapping, orphaned policy, mapping in cert n, `anyPolicy` asserted, unknown critical extension; each expects a specific error |
+| Ratification | Go table test that mutates a valid chain | Flipped signature byte, reordered certs, dropped mapping, orphaned policy, mapping in cert n, `anyPolicy` missing from a cert 2..n or asserted in cert 1, a list node with no elements, unknown critical extension; each expects a specific error |
| Civic | Go table test | Runs before, inside and after an `-effective`/`-expires` window; repeal of each depth; scheduled repeal; stale CRL; CRL signed by the wrong key; `-anchor` pin across a re-ratified version; notes over 200 characters; default-key warning |
| Golden | One checked-in `.gov` | Byte equality, proving determinism |
-| External oracle (optional) | `openssl verify -policy_check -explicit_policy -policy_print`, plus `-crl_check` | OpenSSL builds the same tree and agrees on repeals |
+| External oracle (optional) | `openssl verify -policy_check -explicit_policy -policy 2.5.29.32.0`, plus `-crl_check` | OpenSSL accepts every valid program, rejects broken ones, and agrees on repeals. Without `-policy 2.5.29.32.0` OpenSSL treats the acceptable set as empty |
Ratification tests are Go rather than Lisp because a Lisp source cannot express a broken certificate.
@@ -408,7 +417,7 @@ ### Notes
Comments in the Lisp source travel inside the certs as notes on the clauses they describe, so a `.gov` file documents itself.
-- **Encoding.** Each policy entry may carry RFC 5280 user-notice qualifiers (`id-qt-unotice`, `1.3.6.1.5.5.7.2.2`) whose `explicitText` is a UTF8String. RFC 5280 caps `explicitText` at 200 characters, so a longer note is split across several qualifiers and joined back in order.
+- **Encoding.** Each policy entry may carry RFC 5280 user-notice qualifiers (`id-qt-unotice`, `1.3.6.1.5.5.7.2.2`) whose `explicitText` is a UTF8String. RFC 5280 caps `explicitText` at 200 characters, so a longer note is split across several qualifiers and joined back in order. A node below depth 1 is normally created through `anyPolicy` and would take its (empty) qualifiers, so a node with a note is also listed explicitly in its cert; the explicit entry wins, because RFC 9618 step d.1 runs before d.2, which skips policies already present.
- **Attachment.** The transpiler attaches a `;` comment to the form that begins on the next line, or to the form it trails on the same line. Test annotations (`;=>`, `;=!`) are never notes.
- **Semantics.** Qualifiers do not affect policy processing or evaluation; RFC 5280 only carries them along.
- **Display.** `government tree` prints notes beside nodes. Errors print the note of the innermost noted node, e.g. `E-UNBOUND: no statute for symbol z … (note: "compute the total")`. `government decompile` restores notes as comments, so the round-trip test also covers comments.
@@ -442,6 +451,7 @@ | Cert order in the file? | Anchor first, top-down | Source format |
| OID root? | UUID arc under `2.25` | OID encoding |
| v1 extras? | No variadic lambdas; `if` without else returns `()`; add `strlen`, `strcat`, `str?` | Semantics, builtin table |
| Which X.509 extras? | Effective dates, repeal by CRL, notes, authorship | Civic features |
+| Padding? | Generated by the validator: certs 2..n assert `anyPolicy`, which copies unmapped leaves down. Verified against OpenSSL and Go | Policy tree, encoding |
## Sources
diff --git a/PLAN.md b/PLAN.md
index ed5bb3a4524049636e669b68c4f968a1bec2041a..70b41d2eabef27e66190b1057bbbbb872d86a32d 100644
--- a/PLAN.md
+++ b/PLAN.md
@@ -106,13 +106,13 @@ - Build: `Lower(program []sexpr.Value) *Tree`, which classifies reserved names into ns 0 and numbers nodes in pre-order; `Raise(*Tree) []sexpr.Value`, the inverse, which also keeps a map from each produced value to its node for error locations.
- Learn: an AST as a separate structure from source text; why sibling order must be explicit data once you leave a text format.
- Done when: `Raise(Lower(p))` equals `p` for every `testdata` program, and `'+` raises to the symbol `+`.
16. **Layering** (`ast`)
- - Build: `Layer(*Tree) Layers`, which computes depth n, inserts red-tape chains under shallow atoms and empty lists, and returns per-depth node sets plus parent→child edges.
- - Learn: why the policy tree forces every leaf to depth n (RFC 5280 §6.1.3 step d.3 pruning), and how padding answers that.
- - Done when: the design's worked example `(+ 1 (* 2 3))` produces exactly its nine nodes and the edges in its table.
+ - Build: `Layer(*Tree) Layers`, which computes depth n and returns per-depth node sets plus parent→child edges. No padding: certs 2..n assert `anyPolicy`, and the validator copies each unmapped leaf down.
+ - Learn: why the policy tree forces every leaf to depth n (RFC 5280 §6.1.3 step d.3 pruning), and how `anyPolicy` (RFC 9618 step d.2) makes the validator do the padding.
+ - Done when: the design's worked example `(+ 1 (* 2 3))` produces exactly its seven nodes and the edges in its table.
17. **Unlayering** (`ast`)
- - Build: `Unlayer(Layers) (*Tree, error)`, which rebuilds the tree from depth sets and edges, strips red tape, orders children by id, and enforces decode rules 1–6. Accept a node with several parents (a DAG).
+ - Build: `Unlayer(Layers) (*Tree, error)`, which rebuilds the tree from depth sets and edges, skips copies (a child with its parent's OID), orders children by id, and enforces decode rules 1–6. Accept a node with several parents (a DAG).
- Learn: validating untrusted structure; Phase 5's policy graph will feed this function directly.
- - Done when: `Unlayer(Layer(t))` equals `t` for every `testdata` program, and each broken-structure case (red tape with two children, an atom with a real child, `.` in the wrong place) gives its own error.
+ - Done when: `Unlayer(Layer(t))` equals `t` for every `testdata` program, and each broken-structure case (a copy with a non-copy child, an atom with a real child, a list node with no elements, `.` in the wrong place) gives its own error.
## Phase 3: Building certificates (steps 18–22)
@@ -131,11 +131,11 @@ - Build: encode `PolicyMappings ::= SEQUENCE OF SEQUENCE { issuerDomainPolicy, subjectDomainPolicy }` and attach it as a critical `ExtraExtensions` entry (OID `2.5.29.33`).
- Learn: why Go 1.24 parses this extension but does not write it; what "critical" means for a relying party. Read RFC 5280 §4.2.1.5.
- Done when: `ParseCertificate` returns your mappings in `PolicyMappings` and lists no unhandled critical extensions.
21. **certificatePolicies extension** (`lisp2gov`)
- - Build: hand-encode `certificatePolicies` (OID `2.5.29.32`) as a SEQUENCE of PolicyInformation, OIDs only for now, leaving room for qualifiers in Phase 7.
+ - Build: hand-encode `certificatePolicies` (OID `2.5.29.32`) as a SEQUENCE of PolicyInformation, OIDs only for now, leaving room for qualifiers in Phase 7. Cert 1 lists the top-level nodes; certs 2..n list `anyPolicy` (`2.5.29.32.0`).
- Learn: policy OIDs as assertions a CA makes about a certificate, which is the hook GOVERNMENT abuses. Read RFC 5280 §4.2.1.4.
- Done when: `ParseCertificate` returns the OIDs in `Policies`, including one with the 128-bit `GOV` arc.
22. **The transpiler** (`lisp2gov`, `cmd/government`)
- - Build: `Compile(src) ([]byte, error)`: read, lower, layer, then one cert per depth with that depth's policies and the mappings for its outgoing edges; per-depth keys from the seed; PEM with the source as a header comment. Wire up `government compile`.
+ - Build: `Compile(src) ([]byte, error)`: read, lower, layer, then one cert per depth (cert 1 with the top-level nodes, the rest with `anyPolicy`), each with the mappings for its depth's outgoing edges; per-depth keys from the seed; PEM with the source as a header comment. Wire up `government compile`.
- Learn: how a certification path is a chain of issuer→subject links; how a whole program becomes n+1 signed objects.
- Done when: `government compile examples/sq.lisp -o sq.gov` writes a file that `openssl x509` can print block by block, and compiling twice gives identical bytes. **Milestone: GOVERNMENT programs exist.**
@@ -165,17 +165,17 @@
Implement the algorithm that turns a certificate chain into a program. This is the most important code in the project; build it in two halves so each can be checked against the RFC on its own.
27. **Policy graph without mappings** (`policy`)
- - Build: levels 0..n; the `anyPolicy` root expecting `{anyPolicy}`; RFC 9618 step (d): attach each policy of cert *i* to every depth-*i−1* node expecting it, else to an `anyPolicy` parent; repeated pruning of childless nodes at depth *i−1* or less; step (e) (no extension means an empty graph); explicit policy, so an empty graph fails.
+ - Build: levels 0..n; the `anyPolicy` root expecting `{anyPolicy}`; RFC 9618 step (d): attach each policy of cert *i* to every depth-*i−1* node expecting it, else to an `anyPolicy` parent; step (d)(2): `anyPolicy` in cert *i* creates every expected policy that has no node yet; repeated pruning of childless nodes at depth *i−1* or less; step (e) (no extension means an empty graph); explicit policy, so an empty graph fails.
- Learn: what the valid policy tree/graph means in real PKI (which policies hold along the whole path) and why RFC 9618 replaced the tree with a DAG. Read RFC 5280 §6.1.2–6.1.3 and RFC 9618 §3–5.
- - Done when: hand-built chains where every cert asserts the same policy keep it, a chain where one cert drops it yields failure, and pruning removes a dead-end branch.
+ - Done when: hand-built chains where every cert asserts the same policy keep it, a chain where one cert drops it yields failure, pruning removes a dead-end branch, and an atom at depth 1 survives to depth 3 only when certs 2 and 3 assert `anyPolicy` (the design-phase prototype's `copyonly`/`nocopy` chains).
28. **Policy mappings** (`policy`)
- Build: RFC 5280 §6.1.4 (a)–(b): reject `anyPolicy` in mappings; for each depth-*i* node that is an `issuerDomainPolicy`, set its expected set to the mapped `subjectDomainPolicy` values; skip this step for cert *n*.
- Learn: what mappings mean between real CAs ("your policy X is equivalent to my policy Y"), and why that is exactly a parent→child edge.
- Done when: the design's worked example produces its tree exactly, checked against the table, and a child mapped from two parents gets two parents.
29. **Strict check and limits** (`policy`)
- - Build: after processing, every asserted policy is a node at its depth and every mapping produced an edge (else an orphaned-policy error naming the OID); mappings in cert *n* rejected; `anyPolicy` asserted anywhere rejected; `-max-nodes` budget.
+ - Build: after processing, every asserted policy is a node at its depth, every mapping produced an edge, and node ids are exactly 1..N (else an orphaned-policy error naming the OID); mappings in cert *n* rejected; `anyPolicy` missing from a cert 2..n, or asserted in cert 1, rejected; `-max-nodes` budget.
- Learn: how silent pruning makes RFC processing forgiving, and why a language needs it to be unforgiving; policy-processing DoS (CVE-2026-32281 in Go).
- - Done when: deleting one mapping from a valid file gives a strict-check error, not a shorter program.
+ - Done when: deleting one mapping from a valid file gives an error (a gap in the node ids, or a list node left with no elements), not a shorter program, except for the subtree with the highest ids, which the test documents.
30. **The full pipeline** (`cmd/government`)
- Build: `government run`: load → ratify → policy graph → `Unlayer` → `Raise` → evaluate; errors carry the node OID and depth; exit codes 1 and 2. Switch the test runner to compile every `testdata` file and run the resulting bytes.
- Learn: how the stages compose into a compiler-like pipeline with a trust boundary in the middle.
@@ -228,6 +228,6 @@ - Build: generated programs 200 deep (201 certs) and 1,000 arguments wide; `-max-nodes` tests; one checked-in golden `.gov` compared byte for byte.
- Learn: why reproducible output is a security property (anyone can rebuild and compare), and how golden files catch unintended format changes.
- Done when: stress tests finish in under a few seconds, and the golden test fails if any byte of the encoding changes.
39. **OpenSSL as an oracle** (optional, skipped when `openssl` is absent)
- - Build: run `openssl verify -policy_check -explicit_policy -policy_print -x509_strict` (and `-crl_check` with repeal CRLs) on compiled programs; parse its printed policy tree and compare with ours.
+ - Build: run `openssl verify -policy_check -explicit_policy -policy 2.5.29.32.0 -x509_strict` (and `-crl_check` with repeal CRLs) on compiled programs and on deliberately broken ones. Without `-policy 2.5.29.32.0`, OpenSSL treats the acceptable set as empty and rejects everything; its `-policy_print` shows only top-level policies, so compare accept/reject decisions, not trees.
- Learn: differential testing against an independent implementation; where implementations disagree on edge cases.
- Done when: OpenSSL accepts every valid program, prints the same set of policies at each depth, and rejects each repealed program. **Milestone: done.**