government

commit 1e3679e075694dda43bb3b82dd39f993c8e7705a

tree

parent:
ce159ec40272fb3caed2436702b1426f1550e47d

nmyk <nick@nmyk.io>

2026-11-14T21:17:11-05:00

add hash consing and compile flag to turn it off

diff --git a/DESIGN.md b/DESIGN.md
index b3a22ac612922b5c817a2ee5b3ef0a515b48f6c6..fe3c0ece674a62668c4c2a46d50968282626575b 100644
--- a/DESIGN.md
+++ b/DESIGN.md
@@ -97,7 +97,7 @@ | 5 | A policy matching no parent at depth ≥ 2 is silently dropped | An explicitly listed node (one carrying notes) with a wrong OID would vanish | **Strict check**: every explicitly asserted policy must appear in the final tree |
 | 6 | Mappings in cert n are never processed | Dead data | Rejected |
 | 7 | `anyPolicy` must not be mapped | — | Certs 2..n must assert `anyPolicy`; cert 1 must not (it would create an `anyPolicy` node under the root); it may never be mapped |
 | 8 | An empty tree fails explicit-policy validation | The empty program cannot exist | A program needs ≥ 1 top-level form |
-| 9 | One child OID mapped from several parents gets several parents (9618) or is duplicated (5280) | Shared subexpressions | Allowed; evaluated directly on the DAG, never unfolded (decided) |
+| 9 | One child OID mapped from several parents gets several parents (9618) or is duplicated (5280) | Shared subexpressions | The compiler **shares** identical lists at the same depth by default (`--no-share` keeps copies); the decoder and evaluator work on the DAG directly, never unfolding it |
 
 **Strict check.** Plain RFC processing forgives mistakes by deleting code. After policy processing, the interpreter also verifies that certs 2..n assert `anyPolicy` and cert 1 does not, that every explicitly asserted policy in cert *i* is a node at depth *i*, that every mapping in cert *i* produced an edge, and that the program's distinct node ids are exactly 1..N with no gaps. A failure is a ratification error naming the orphaned OID. This turns silent data loss into a compile-style error.
 
@@ -109,7 +109,7 @@ Every node OID is `GOV.<namespace>.<node-id>.<payload…>`. For a builtin the payload is one arc, its fixed code from the builtin table. For every other node it is the Unicode code points of the node's text, one arc each (so ASCII `A` is arc 65).
 
 **Root arc.** `GOV = 2.25.9665536045854453410170579537759926404`. The `2.25` arc takes a UUID as an integer and needs no registration (ITU-T X.667); this one is UUID `074583b8-3bb7-4179-8669-827877697884`. Go's `x509.ParseOID` handles the 128-bit arc. Tools print `GOV` as shorthand.
 
-**Node id.** A positive integer, unique across the whole program, assigned in pre-order by the transpiler. Siblings are ordered by ascending id. Uniqueness is what makes constraints 2 and 3 above go away.
+**Node id.** A positive integer, unique across the whole program, assigned in pre-order by the transpiler. Siblings are ordered by ascending id. Uniqueness is what makes constraints 2 and 3 above go away. Sharing renumbers ids so that every list a shared node belongs to still sorts in argument order (see Shared subexpressions).
 
 **Payload.** For symbols, numbers and strings: Unicode code points 0–0x10FFFF, excluding surrogates. ASCII is the common case. Verified locally: an arc of 128512 (😀) survives a create/parse round trip.
 
@@ -333,7 +333,7 @@ | `government prog.gov` | Same as `government run prog.gov`, as with `python prog.py`. A command name always wins over a file of the same name, and an argument that is neither a file nor a `.gov` name is reported as an unknown command |
 | `government verify prog.gov` | Ratify and decode only; print the certificate of ratification; exit 0 if the program is lawful |
 | `government tree prog.gov` | Print the policy tree, with `GOV` abbreviated, payloads decoded and notes shown |
 | `government decompile prog.gov` | Print equivalent Lisp source, notes restored as comments |
-| `government compile prog.gov.lisp -o prog.gov` | Run the transpiler (`--key`, `--author`, `--seed`, `--effective`, `--expires`, `--name`, `--source`) |
+| `government compile prog.gov.lisp -o prog.gov` | Run the transpiler (`--key`, `--author`, `--seed`, `--effective`, `--expires`, `--name`, `--source`, `--no-share`) |
 | `government repeal prog.gov -o repeal.crl` | Write a CRL repealing the program (`--key` or `--seed`, `--reason`, `--at`, `--issued`, `--next-update`; `--next-update` defaults to 9999-12-31, a permanent repeal) |
 
 Shared flags: `--at <RFC 3339 time>`, `--anchor <cert or public key PEM>`, `--crl <file>` (repeatable), `--max-nodes <n>`.
@@ -351,12 +351,25 @@
 1. **Read** Lisp: integers, floats, strings (escapes `\"` `\\` `\n` `\t` `\u{…}`), symbols, `T`/`F`, `'x`, `;` comments (kept as notes), dotted pairs.
 2. **Classify**: reserved names go to ns 0, other symbols to ns 1, literals to ns 2–4. Expand `'x` to `(quote x)`. Attach each comment to its form.
 3. **Number** nodes in pre-order, starting at 1.
-4. **Layer**: compute n = max depth and each node's depth. No padding is written: `anyPolicy` makes the validator copy shallow leaves down.
-5. **Assign**: cert 1 lists the top-level nodes; certs 2..n list `anyPolicy`, plus any node that carries notes; each parent→child edge becomes a mapping in the parent's cert.
-6. **Sign**: take the author key (or the default seed), derive one Ed25519 key per depth, build the anchor and certs 1..n, and hand-encode certificatePolicies and policyMappings.
-7. **Emit** PEM, with the original source above the first block if `--source` is given.
+4. **Share** (unless `--no-share`): merge identical lists at the same depth into one node with several parents, and renumber. See Shared subexpressions.
+5. **Layer**: compute n = max depth and each node's depth. No padding is written: `anyPolicy` makes the validator copy shallow leaves down.
+6. **Assign**: cert 1 lists the top-level nodes; certs 2..n list `anyPolicy`, plus any node that carries notes; each parent→child edge becomes a mapping in the parent's cert.
+7. **Sign**: take the author key (or the default seed), derive one Ed25519 key per depth, build the anchor and certs 1..n, and hand-encode certificatePolicies and policyMappings.
+8. **Emit** PEM, with the original source above the first block if `--source` is given.
 
 The same source and flags always give the same bytes.
+
+### Shared subexpressions
+
+RFC 5280's tree would have to duplicate a subexpression used in two places; RFC 9618's graph lets one node have several parents. So by default the compiler hash-conses: every subtree gets a number for its shape (node kinds, text and notes, recursively), and a list whose shape already appeared at the same depth becomes that earlier node, with one more parent. A shared node costs one mapping per parent, as before, but its own subtree is written once, so sharing never makes a program bigger. `(+ (* 1 E) (- E 0))` nested 20 deep is about a million nodes as a tree and 162 shared.
+
+Three rules keep the meaning intact:
+
+- **Only lists are shared.** An atom costs one mapping per parent either way, so sharing it saves nothing.
+- **Equal siblings stay apart.** A cert's policies are a set, so in `(+ (f) (f))` one shared `(f)` would lose an argument. The k-th of several equal siblings is shared only with other k-th copies.
+- **Argument order must survive.** Children sort by id in every parent, and two parents can disagree: `(a (g) (h))` and `(b (h) (g))` at one depth would need `(g)` both before and after `(h)`. Before sharing, the compiler checks that the node can follow its previous sibling without contradicting an order already required, and keeps a separate copy if not. Ids are then renumbered 1..N in an order every parent agrees with, ties going to the original pre-order id, so a program with nothing to share compiles to exactly the bytes it would without sharing.
+
+Notes are part of a subtree's shape, so no comment is lost. The decoder needs nothing new: `Unlayer` keeps a node with several parents as one node, and `FromTree` turns it into one value shared by several lists, so the program in memory is as small as the one in the certificates. Evaluation never mutates code, so it cannot tell the sharing apart from copies. A runtime error inside a shared node names that one node; the note `Locate` reports comes from the first parent that has one.
 
 ### Test file format
 
@@ -456,7 +469,7 @@ | --- | --- | --- |
 | What counts as true? | Scheme-style: only `F` is false | Semantics |
 | Number size? | 64 bits: int64 (overflow is an error) and float64 | Semantics |
 | What does a program output? | Value of every non-`define` top-level form | Semantics, tests |
-| Shared subexpressions? | Allowed; evaluated on the DAG without unfolding | Policy tree |
+| Shared subexpressions? | Yes, by default (`--no-share` to keep copies); evaluated on the DAG without unfolding | Shared subexpressions |
 | `define` in lambda bodies? | Allowed at the start of the body; sequential, no mutual recursion | Semantics |
 | Who is trusted? | The file's own anchor (`--anchor` optional) | Source format |
 | Should programs expire? | Only when built with `--expires` | Civic features |
diff --git a/PLAN.md b/PLAN.md
index 7ff5890b3de9ef96347cd1b8fc3bbcb83426dbc4..4c3789d1d24be58124b200d824d84727905879fe 100644
--- a/PLAN.md
+++ b/PLAN.md
@@ -220,9 +220,9 @@
 Finish with the tests that prove GOVERNMENT is deterministic, survives large programs, and agrees with an independent PKI implementation.
 
 37. **Shared subexpressions** (`lisp2gov`, `eval`)
-    - Build: an opt-in `--share` compile flag that hash-conses identical subtrees at the same depth into one node with several parents; confirm `Unlayer` and the evaluator walk the DAG without copying.
+    - Build: sharing by default, with an opt-out `--no-share` compile flag: hash-cons identical subtrees at the same depth into one node with several parents; confirm `Unlayer` and the evaluator walk the DAG without copying.
     - Learn: hash-consing; how RFC 9618's graph makes sharing cheap where RFC 5280's tree would duplicate it.
-    - Done when: every `testdata` file gives the same results with and without `--share`, and a program with 20 nested copies of one subexpression stays small.
+    - Done when: every `testdata` file gives the same results with and without `--no-share`, and a program with 20 nested copies of one subexpression stays small.
 38. **Stress and golden tests**
     - Build: generated programs 200 deep (201 certs) and 1,000 arguments wide; `--max-nodes` tests; one checked-in golden `.gov` compared byte for byte.
     - Learn: why reproducible output is a security property (anyone can rebuild and compare), and how golden files catch unintended format changes.
diff --git a/ast/share.go b/ast/share.go
new file mode 100644
index 0000000000000000000000000000000000000000..f15fd651a3e3339ff570c409b93ad3f627be13e9
--- /dev/null
+++ b/ast/share.go
@@ -0,0 +1,199 @@
+package ast
+
+import (
+	"cmp"
+	"container/heap"
+	"encoding/binary"
+)
+
+// Share merges identical subtrees at the same depth into one node with
+// several parents, which RFC 9618's policy graph represents directly: a
+// shared node costs one mapping per parent, and its own subtree is
+// written once. Nothing is shared that would change the program, so the
+// result evaluates exactly as t does.
+//
+// Only lists are shared. An atom costs one mapping per parent either way,
+// so sharing one saves nothing, and sharing atoms would make argument
+// order impossible to keep far more often (see below).
+//
+// Three rules keep the meaning intact:
+//
+//   - A subtree is identical only with its notes, so no comment is lost.
+//
+//   - Two equal children of one parent stay two nodes: a cert's policies
+//     are a set, so (+ (f) (f)) sharing one (f) would lose an argument.
+//     The k-th copy among siblings is shared only with other k-th copies.
+//
+//   - Ids still have to give argument order: within every parent, children
+//     sort by id. Sharing adds a node to several parents, and their orders
+//     can disagree: (f (g) (h)) and (k (h) (g)) at one depth would need
+//     (g) both before and after (h). Before sharing, Share checks the
+//     node can follow the sibling before it without such a contradiction,
+//     and keeps a separate copy when it cannot.
+//
+// Ids are then renumbered 1..N in an order every parent agrees with,
+// breaking ties by original id, so a program with nothing to share keeps
+// exactly the ids Build gave it.
+//
+// t must be as Build made it: ids 1..N, each node in one place.
+func Share(t *Tree) *Tree {
+	s := &sharer{
+		interned: map[string]int{},
+		canon:    map[shareKey]*Node{},
+		orig:     map[*Node]uint64{},
+		after:    map[*Node][]*Node{},
+	}
+	out := &Tree{Forms: s.children(t.Forms, 1)}
+	s.renumber()
+	return out
+}
+
+type shareKey struct {
+	depth, rank, structure int
+}
+
+type sharer struct {
+	structure []int          // input node id -> its structure's number + 1
+	interned  map[string]int // a structure's description -> its number
+	key       []byte
+	canon     map[shareKey]*Node // the first output node for each key
+	orig      map[*Node]uint64   // output node -> id of the input node it came from
+	after     map[*Node][]*Node  // order constraints: a must come before each of after[a]
+	nodes     []*Node
+}
+
+// structureOf numbers each distinct subtree shape, notes included, so two
+// subtrees are identical exactly when their numbers match.
+func (s *sharer) structureOf(n *Node) int {
+	if int(n.ID) < len(s.structure) && s.structure[n.ID] > 0 {
+		return s.structure[n.ID] - 1
+	}
+	children := make([]int, len(n.Children))
+	for i, c := range n.Children {
+		children[i] = s.structureOf(c)
+	}
+	k := append(s.key[:0], byte(n.NS))
+	k = binary.AppendUvarint(k, uint64(len(n.Text)))
+	k = append(k, n.Text...)
+	k = binary.AppendUvarint(k, uint64(len(n.Note)))
+	k = append(k, n.Note...)
+	for _, c := range children {
+		k = binary.AppendUvarint(k, uint64(c))
+	}
+	s.key = k
+	id, ok := s.interned[string(k)]
+	if !ok {
+		id = len(s.interned)
+		s.interned[string(k)] = id
+	}
+	for int(n.ID) >= len(s.structure) {
+		s.structure = append(s.structure, 0)
+	}
+	s.structure[n.ID] = id + 1
+	return id
+}
+
+// children places the elements of one list (or the top-level forms) at
+// depth, sharing where the rules allow, and records that each must come
+// after the one before it.
+func (s *sharer) children(in []*Node, depth int) []*Node {
+	var out []*Node
+	rank := map[int]int{}
+	var prev *Node
+	for _, c := range in {
+		st := s.structureOf(c)
+		key := shareKey{depth, rank[st], st}
+		rank[st]++
+
+		n := s.canon[key]
+		if n == nil || (prev != nil && s.reaches(n, prev)) {
+			n = s.copy(c, depth)
+			if c.IsList() && s.canon[key] == nil {
+				s.canon[key] = n
+			}
+		}
+		if prev != nil {
+			s.after[prev] = append(s.after[prev], n)
+		}
+		out = append(out, n)
+		prev = n
+	}
+	return out
+}
+
+func (s *sharer) copy(in *Node, depth int) *Node {
+	n := &Node{NS: in.NS, Text: in.Text, Note: in.Note}
+	s.orig[n] = in.ID
+	s.nodes = append(s.nodes, n)
+	n.Children = s.children(in.Children, depth+1)
+	return n
+}
+
+// reaches reports whether from must already come before to. Ordering to
+// before from as well would then be a contradiction.
+func (s *sharer) reaches(from, to *Node) bool {
+	seen := map[*Node]bool{}
+	stack := []*Node{from}
+	for len(stack) > 0 {
+		n := stack[len(stack)-1]
+		stack = stack[:len(stack)-1]
+		if n == to {
+			return true
+		}
+		if seen[n] {
+			continue
+		}
+		seen[n] = true
+		stack = append(stack, s.after[n]...)
+	}
+	return false
+}
+
+// renumber gives the nodes ids 1..N in an order that puts every list's
+// elements in argument order. Among the nodes free to go next it picks
+// the one with the smallest original id.
+func (s *sharer) renumber() {
+	waiting := map[*Node]int{}
+	for _, n := range s.nodes {
+		for _, m := range s.after[n] {
+			waiting[m]++
+		}
+	}
+	ready := &byOrig{orig: s.orig}
+	for _, n := range s.nodes {
+		if waiting[n] == 0 {
+			heap.Push(ready, n)
+		}
+	}
+	var next uint64
+	for ready.Len() > 0 {
+		n := heap.Pop(ready).(*Node)
+		next++
+		n.ID = next
+		for _, m := range s.after[n] {
+			if waiting[m]--; waiting[m] == 0 {
+				heap.Push(ready, m)
+			}
+		}
+	}
+	if int(next) != len(s.nodes) {
+		// children never adds an order that closes a cycle, so every node
+		// is numbered.
+		panic("ast.Share: argument orders contradict each other")
+	}
+}
+
+type byOrig struct {
+	nodes []*Node
+	orig  map[*Node]uint64
+}
+
+func (h *byOrig) Len() int           { return len(h.nodes) }
+func (h *byOrig) Less(i, j int) bool { return cmp.Less(h.orig[h.nodes[i]], h.orig[h.nodes[j]]) }
+func (h *byOrig) Swap(i, j int)      { h.nodes[i], h.nodes[j] = h.nodes[j], h.nodes[i] }
+func (h *byOrig) Push(x any)         { h.nodes = append(h.nodes, x.(*Node)) }
+func (h *byOrig) Pop() any {
+	n := h.nodes[len(h.nodes)-1]
+	h.nodes = h.nodes[:len(h.nodes)-1]
+	return n
+}
diff --git a/ast/share_test.go b/ast/share_test.go
new file mode 100644
index 0000000000000000000000000000000000000000..d133026c71cada2813fd1dc6d3cb07a4ff7dd897
--- /dev/null
+++ b/ast/share_test.go
@@ -0,0 +1,191 @@
+package ast
+
+import (
+	"os"
+	"path/filepath"
+	"reflect"
+	"strings"
+	"testing"
+
+	"government/sexpr"
+)
+
+func shareSource(t *testing.T, src string) (built, shared *Tree) {
+	t.Helper()
+	datums, err := sexpr.ReadProgram(src)
+	if err != nil {
+		t.Fatal(err)
+	}
+	built, err = Build(datums)
+	if err != nil {
+		t.Fatal(err)
+	}
+	again, _ := Build(datums)
+	return built, Share(again)
+}
+
+// checkShared checks everything the certificates need from a shared tree,
+// and that it still means src.
+func checkShared(t *testing.T, src string, tree *Tree) {
+	t.Helper()
+	seen := map[*Node]bool{}
+	var ids []uint64
+	var walk func(n *Node)
+	walk = func(n *Node) {
+		if seen[n] {
+			return
+		}
+		seen[n] = true
+		ids = append(ids, n.ID)
+		parentOf := map[*Node]bool{}
+		for i, c := range n.Children {
+			if parentOf[c] {
+				t.Errorf("%s has %s as a child twice", n, c)
+			}
+			parentOf[c] = true
+			if i > 0 && n.Children[i-1].ID >= c.ID {
+				t.Errorf("%s: children out of id order: %s before %s", n, n.Children[i-1], c)
+			}
+			walk(c)
+		}
+	}
+	for i, f := range tree.Forms {
+		if i > 0 && tree.Forms[i-1].ID >= f.ID {
+			t.Errorf("forms out of id order: %s before %s", tree.Forms[i-1], f)
+		}
+		walk(f)
+	}
+	want := make([]bool, len(ids)+1)
+	for _, id := range ids {
+		if id < 1 || int(id) > len(ids) || want[id] {
+			t.Fatalf("ids are not exactly 1..%d: %v", len(ids), ids)
+		}
+		want[id] = true
+	}
+
+	// Through the certificates' representation and back.
+	layers, err := Layer(tree)
+	if err != nil {
+		t.Fatal(err)
+	}
+	decoded, err := Unlayer(layers)
+	if err != nil {
+		t.Fatal(err)
+	}
+	forms, _, err := FromTree(decoded)
+	if err != nil {
+		t.Fatal(err)
+	}
+	orig, err := sexpr.ReadAll(src)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if !reflect.DeepEqual(forms, orig) {
+		t.Errorf("shared program reads back as\n%v\nwant\n%v", printAll(forms), printAll(orig))
+	}
+}
+
+func countNodes(tree *Tree) int {
+	seen := map[*Node]bool{}
+	var walk func(n *Node)
+	walk = func(n *Node) {
+		if !seen[n] {
+			seen[n] = true
+			for _, c := range n.Children {
+				walk(c)
+			}
+		}
+	}
+	for _, f := range tree.Forms {
+		walk(f)
+	}
+	return len(seen)
+}
+
+func TestShare(t *testing.T) {
+	tests := []struct {
+		name, src string
+		saved     int // nodes the shared tree has fewer of
+	}{
+		{"nothing repeated", "(+ 1 (* 2 3))", 0},
+		{"atoms are never shared", "(+ x x) (- x x)", 0},
+		{"one list, two parents", "(g (f 1)) (h (f 1))", 3},
+		{"equal siblings stay apart", "(+ (f 1) (f 1))", 0},
+		{"k-th copies share with k-th copies", "(+ (f 1) (f 1)) (- (f 1) (f 1))", 6},
+		{"different depths stay apart", "(g (f 1)) (f 1)", 0},
+		{"different notes stay apart", "(g\n (f 1) ; one\n)\n(h\n (f 1) ; two\n)\n", 0},
+		{"equal notes share", "(g\n (f 1) ; same\n)\n(h\n (f 1) ; same\n)\n", 3},
+		{"a shared list brings its subtree", "(a (b (c (d 1)))) (e (b (c (d 1))))", 7},
+		{"argument orders that disagree", "(a (g) (h)) (b (h) (g))", 2},
+		{"dotted tails", "(a (f . g)) (b (f . g))", 4},
+		{"top-level forms never merge", "(f 1) (f 1)", 0},
+	}
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			built, shared := shareSource(t, tt.src)
+			checkShared(t, tt.src, shared)
+			if got := countNodes(built) - countNodes(shared); got != tt.saved {
+				t.Errorf("saved %d nodes, want %d", got, tt.saved)
+			}
+		})
+	}
+}
+
+// TestShareChangesNothingWithoutRepeats: with nothing to share, Share must
+// keep Build's ids, so such programs compile to the same bytes as before.
+func TestShareChangesNothingWithoutRepeats(t *testing.T) {
+	src := "(define sq (lambda (n) (* n n)))\n(sq (+ 1 2))\n'(a . b)\n"
+	built, shared := shareSource(t, src)
+	if !reflect.DeepEqual(built, shared) {
+		t.Error("Share changed a program with nothing to share")
+	}
+}
+
+func TestShareKeepsNodesShared(t *testing.T) {
+	_, shared := shareSource(t, "(g (f 1)) (h (f 1))")
+	if shared.Forms[0].Children[1] != shared.Forms[1].Children[1] {
+		t.Fatal("(f 1) was not shared")
+	}
+	layers, _ := Layer(shared)
+	decoded, err := Unlayer(layers)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if decoded.Forms[0].Children[1] != decoded.Forms[1].Children[1] {
+		t.Error("Unlayer unfolded the shared node")
+	}
+	forms, _, err := FromTree(decoded)
+	if err != nil {
+		t.Fatal(err)
+	}
+	a := forms[0].(*sexpr.Pair).Cdr.(*sexpr.Pair).Car
+	b := forms[1].(*sexpr.Pair).Cdr.(*sexpr.Pair).Car
+	if a != b {
+		t.Error("FromTree unfolded the shared node into two lists")
+	}
+}
+
+// TestShareTestdata runs Share over every test program, which between them
+// repeat plenty of subexpressions.
+func TestShareTestdata(t *testing.T) {
+	paths, _ := filepath.Glob("../testdata/*.gov.lisp")
+	examples, _ := filepath.Glob("../examples/*.gov.lisp")
+	for _, path := range append(paths, examples...) {
+		src, err := os.ReadFile(path)
+		if err != nil {
+			t.Fatal(err)
+		}
+		t.Run(strings.TrimSuffix(filepath.Base(path), ".gov.lisp"), func(t *testing.T) {
+			_, shared := shareSource(t, string(src))
+			checkShared(t, string(src), shared)
+		})
+	}
+}
+
+func printAll(forms []sexpr.Value) string {
+	var lines []string
+	for _, f := range forms {
+		lines = append(lines, sexpr.Print(f))
+	}
+	return strings.Join(lines, "\n")
+}
diff --git a/ast/tree.go b/ast/tree.go
index 9a0d7149f19461c2fa6ed091a127ac771f6f4e1d..f9397a942f5d4b2e9a3759e017634a7bce9d5b3e 100644
--- a/ast/tree.go
+++ b/ast/tree.go
@@ -121,25 +121,46 @@ type Origins map[*sexpr.Pair]*Node
 
 // FromTree is ToTree's inverse. A builtin name becomes the plain symbol, so
 // '+ quotes the symbol +, exactly as in source.
+//
+// A node shared by several parents becomes one value shared by several
+// lists, so a shared subexpression is never unfolded: the program in memory
+// is as small as the one in the certificates. Values are immutable, so
+// nothing can tell the sharing apart from copies.
 func FromTree(t *Tree) ([]sexpr.Value, Origins, error) {
-	origins := Origins{}
+	c := &converter{origins: Origins{}, done: map[*Node]sexpr.Value{}}
 	forms := make([]sexpr.Value, len(t.Forms))
 	for i, n := range t.Forms {
-		v, err := value(n, origins)
+		v, err := c.value(n)
 		if err != nil {
 			return nil, nil, err
 		}
 		forms[i] = v
 	}
-	return forms, origins, nil
+	return forms, c.origins, nil
+}
+
+type converter struct {
+	origins Origins
+	done    map[*Node]sexpr.Value
+}
+
+func (c *converter) value(n *Node) (sexpr.Value, error) {
+	if v, ok := c.done[n]; ok {
+		return v, nil
+	}
+	v, err := c.convert(n)
+	if err == nil {
+		c.done[n] = v
+	}
+	return v, err
 }
 
-func value(n *Node, origins Origins) (sexpr.Value, error) {
+func (c *converter) convert(n *Node) (sexpr.Value, error) {
 	if err := checkShape(n); err != nil {
 		return nil, err
 	}
 	if n.IsList() {
-		return listValue(n, origins)
+		return c.listValue(n)
 	}
 	switch n.NS {
 	case NSBuiltin:
@@ -168,26 +189,26 @@ 	}
 	return nil, fmt.Errorf("%s: unknown %s", n, n.NS)
 }
 
-func listValue(n *Node, origins Origins) (sexpr.Value, error) {
+func (c *converter) listValue(n *Node) (sexpr.Value, error) {
 	children := n.Children
 	var tail sexpr.Value = sexpr.Nil
 	if k := len(children); k >= 3 && isDot(children[k-2]) {
-		t, err := value(children[k-1], origins)
+		t, err := c.value(children[k-1])
 		if err != nil {
 			return nil, err
 		}
 		tail, children = t, children[:k-2]
 	}
 	items := make([]sexpr.Value, len(children))
-	for i, c := range children {
-		v, err := value(c, origins)
+	for i, child := range children {
+		v, err := c.value(child)
 		if err != nil {
 			return nil, err
 		}
 		items[i] = v
 	}
 	list := sexpr.ListWithTail(tail, items...).(*sexpr.Pair)
-	origins[list] = n
+	c.origins[list] = n
 	return list, nil
 }
 
diff --git a/cmd/government/compile.go b/cmd/government/compile.go
index d4b71ae1434f1bb93c057d3e68e739b65f22d2f8..18cc7f4ae45a633540682400a04e18dc849bca6e 100644
--- a/cmd/government/compile.go
+++ b/cmd/government/compile.go
@@ -18,7 +18,7 @@
 func newCompileCmd() *cobra.Command {
 	var (
 		out, name, keyFile, seedHex, author string
-		withSource                          bool
+		withSource, noShare                 bool
 		effective, expires                  dateFlag
 	)
 	cmd := &cobra.Command{
@@ -40,7 +40,7 @@ 				if name == base {
 					name = strings.TrimSuffix(base, filepath.Ext(base))
 				}
 			}
-			opts := lisp2gov.Options{Name: name, IncludeSource: withSource, Author: author, Effective: effective.t, Expires: expires.t}
+			opts := lisp2gov.Options{Name: name, IncludeSource: withSource, NoShare: noShare, Author: author, Effective: effective.t, Expires: expires.t}
 			if opts.Seed, err = signingSeed(keyFile, seedHex); err != nil {
 				return refused("%v", err)
 			}
@@ -59,6 +59,7 @@ 	f := cmd.Flags()
 	f.StringVarP(&out, "output", "o", "", "write the program to `file` instead of standard output")
 	f.StringVar(&name, "name", "", "program `name` for the certs' O= field (default: the source file's name without .gov.lisp)")
 	f.BoolVar(&withSource, "source", false, "copy the GovLisp source into the file's header")
+	f.BoolVar(&noShare, "no-share", false, "keep every copy of a repeated subexpression instead of sharing one node between them")
 	f.StringVar(&keyFile, "key", "", "sign with the Ed25519 PKCS#8 private key in `file` (make one with: openssl genpkey -algorithm ed25519)")
 	f.StringVar(&seedHex, "seed", "", "sign with keys derived from this 32-byte `hex` seed instead of a key file")
 	f.StringVar(&author, "author", "", "name the `author` in the anchor, as \"Name <email>\"")
diff --git a/cmd/government/main_test.go b/cmd/government/main_test.go
index bb325486aa9a954ae1d41bf78393e159a9d00b25..5abf531f958d96095d87471c190834c465871ea2 100644
--- a/cmd/government/main_test.go
+++ b/cmd/government/main_test.go
@@ -115,3 +115,31 @@ 			t.Errorf("%s --help: exit %d, %q", cmd, code, stdout)
 		}
 	}
 }
+
+// TestCompileShares: sharing is the default and --no-share turns it off;
+// both run the same, and the shared program is the smaller one.
+func TestCompileShares(t *testing.T) {
+	src := writeSource(t, "twice", "(define sq (lambda (n) (* n n)))\n(+ (sq (+ 1 2)) (- (sq (+ 1 2)) 1))\n(* (sq (+ 1 2)) 2)\n")
+	dir := t.TempDir()
+	shared, full := filepath.Join(dir, "shared.gov"), filepath.Join(dir, "full.gov")
+	if code, _, stderr := runCmd("compile", src, "-o", shared); code != exitOK {
+		t.Fatal(stderr)
+	}
+	if code, _, stderr := runCmd("compile", src, "-o", full, "--no-share"); code != exitOK {
+		t.Fatal(stderr)
+	}
+	_, a, _ := runCmd(shared)
+	_, b, _ := runCmd(full)
+	if a != "17\n18\n" || a != b {
+		t.Errorf("shared printed %q, unshared %q; want \"17\\n18\\n\" from both", a, b)
+	}
+	x, _ := os.ReadFile(shared)
+	y, _ := os.ReadFile(full)
+	if len(x) >= len(y) {
+		t.Errorf("shared program is %d bytes, unshared %d", len(x), len(y))
+	}
+	_, tree, _ := runCmd("tree", shared)
+	if !strings.Contains(tree, "(shared, drawn above)") {
+		t.Errorf("tree shows no shared node:\n%s", tree)
+	}
+}
diff --git a/lisp2gov/compile.go b/lisp2gov/compile.go
index b824a4a9361fbb22c8b122c1c693f47249b2988f..e830b580059071cd6871998147afde93b56f04f9 100644
--- a/lisp2gov/compile.go
+++ b/lisp2gov/compile.go
@@ -22,6 +22,10 @@ 	// IncludeSource copies the Lisp source into the file's header. It is
 	// off by default: the certs alone are the program, and the source
 	// may hold comments or code the author does not want to publish.
 	IncludeSource bool
+	// NoShare keeps every copy of a repeated subexpression. By default
+	// identical lists at the same depth become one node with several
+	// parents (see ast.Share), which only ever makes a program smaller.
+	NoShare bool
 	// Author is who ratifies the program, as "Name <email>", "Name" or
 	// "email". The name goes in the anchor's subject and the email in its
 	// subjectAltName. It is only a claim unless Seed is the author's own
@@ -105,6 +109,9 @@ 	}
 	tree, err := ast.Build(datums)
 	if err != nil {
 		return nil, err
+	}
+	if !opts.NoShare {
+		tree = ast.Share(tree)
 	}
 	layers, err := ast.Layer(tree)
 	if err != nil {
diff --git a/lisp2gov/share_test.go b/lisp2gov/share_test.go
new file mode 100644
index 0000000000000000000000000000000000000000..e1792f5fe02f2de8f6cc37bcfa5ccc1e9ccd7780
--- /dev/null
+++ b/lisp2gov/share_test.go
@@ -0,0 +1,120 @@
+package lisp2gov_test
+
+import (
+	"bytes"
+	"fmt"
+	"strings"
+	"testing"
+	"time"
+
+	"government/lisp2gov"
+	"government/program"
+	"government/sexpr"
+)
+
+// doubling nests k copies deep: each level uses the one below twice, so
+// written out as a tree the program has about 2^k leaves, and its value
+// is 2^k.
+func doubling(k int) string {
+	e := "1"
+	for range k {
+		e = "(+ (* 1 " + e + ") (- " + e + " 0))"
+	}
+	return e
+}
+
+// distinctPairs counts the cons cells reachable from the forms, each once,
+// which is how much memory the decoded program really holds.
+func distinctPairs(forms []sexpr.Value) int {
+	seen := map[*sexpr.Pair]bool{}
+	var walk func(v sexpr.Value)
+	walk = func(v sexpr.Value) {
+		p, ok := v.(*sexpr.Pair)
+		if !ok || seen[p] {
+			return
+		}
+		seen[p] = true
+		walk(p.Car)
+		walk(p.Cdr)
+	}
+	for _, f := range forms {
+		walk(f)
+	}
+	return len(seen)
+}
+
+func load(t *testing.T, src string, opts lisp2gov.Options) ([]byte, *program.Program) {
+	t.Helper()
+	data, err := lisp2gov.Compile(src, opts)
+	if err != nil {
+		t.Fatal(err)
+	}
+	p, err := program.Load(data, program.Options{At: time.Now()})
+	if err != nil {
+		t.Fatal(err)
+	}
+	return data, p
+}
+
+func run(t *testing.T, p *program.Program) string {
+	t.Helper()
+	var out bytes.Buffer
+	if err := p.Run(&out); err != nil {
+		t.Fatal(err)
+	}
+	return strings.TrimSpace(out.String())
+}
+
+// TestNestedCopiesStaySmall: shared, a program that is a million nodes as
+// a tree compiles to a couple of hundred, decodes to as many lists in
+// memory, and still evaluates to the right answer.
+//
+// At 20 levels the source alone is 18 MB, the 2^20 copies written out, and
+// reading it takes several seconds; -short stops at 12.
+func TestNestedCopiesStaySmall(t *testing.T) {
+	for _, k := range []int{12, 20} {
+		if k > 12 && testing.Short() {
+			t.Skip("20 levels is an 18 MB source")
+		}
+		data, p := load(t, doubling(k), lisp2gov.Options{})
+		if p.Nodes > 8*k+2 {
+			t.Errorf("%d levels: %d nodes; sharing should keep it near 8 per level", k, p.Nodes)
+		}
+		if len(data) > 5000*k {
+			t.Errorf("%d levels: %d bytes", k, len(data))
+		}
+		// Three lists of three elements per level: nine cons cells.
+		if n := distinctPairs(p.Forms); n > 9*k+3 {
+			t.Errorf("%d levels: decoded program holds %d cons cells; the DAG was unfolded", k, n)
+		}
+		if got, want := run(t, p), fmt.Sprint(1<<k); got != want {
+			t.Errorf("%d levels: value %s, want %s", k, got, want)
+		}
+	}
+}
+
+// TestSharingOnlyEverShrinks compares the two modes on the same programs:
+// identical results, and the shared one never bigger.
+func TestSharingOnlyEverShrinks(t *testing.T) {
+	for _, src := range []string{
+		"(+ 1 (* 2 3))",
+		doubling(6),
+		"(define f (lambda (x) (cons (car x) (car x))))\n(f '(1 2))\n(cons (car '(1 2)) (cdr '(1 2)))",
+	} {
+		shared, sp := load(t, src, lisp2gov.Options{})
+		full, fp := load(t, src, lisp2gov.Options{NoShare: true})
+		if a, b := run(t, sp), run(t, fp); a != b {
+			t.Errorf("%s: %q shared, %q not", src, a, b)
+		}
+		if len(shared) > len(full) || sp.Nodes > fp.Nodes {
+			t.Errorf("%s: shared is %d bytes and %d nodes, unshared %d bytes and %d nodes",
+				src, len(shared), sp.Nodes, len(full), fp.Nodes)
+		}
+	}
+	_, sp := load(t, doubling(6), lisp2gov.Options{})
+	_, fp := load(t, doubling(6), lisp2gov.Options{NoShare: true})
+	t.Logf("doubling(6): %d nodes shared, %d not", sp.Nodes, fp.Nodes)
+	if sp.Nodes*4 > fp.Nodes {
+		t.Errorf("doubling(6) shared %d nodes of %d; expected far fewer", sp.Nodes, fp.Nodes)
+	}
+}
diff --git a/lisp_test.go b/lisp_test.go
index 0fb2dfb7ad899223b066c547f15288ca2d183c67..8eabeb63911896af7795827c13516be6731b7d30 100644
--- a/lisp_test.go
+++ b/lisp_test.go
@@ -14,9 +14,9 @@ 	"government/program"
 )
 
 // TestLispFiles compiles every testdata/*.gov.lisp file to a .gov program,
-// runs that program through the whole pipeline (ratify, policy graph,
-// decode, evaluate), and checks each form against the expectation written
-// in the source.
+// with and without shared subexpressions, runs that program through the
+// whole pipeline (ratify, policy graph, decode, evaluate), and checks each
+// form against the expectation written in the source.
 func TestLispFiles(t *testing.T) {
 	paths, err := filepath.Glob("testdata/*.gov.lisp")
 	if err != nil {
@@ -26,33 +26,41 @@ 	if len(paths) == 0 {
 		t.Fatal("no testdata/*.gov.lisp files found")
 	}
 	for _, path := range paths {
-		t.Run(filepath.Base(path), func(t *testing.T) {
-			src, err := os.ReadFile(path)
-			if err != nil {
-				t.Fatal(err)
-			}
-			f, err := lisptest.Parse(path, string(src))
-			if err != nil {
-				t.Fatal(err)
-			}
-			data, err := lisp2gov.Compile(string(src), lisp2gov.Options{})
-			if err != nil {
-				t.Fatal(err)
-			}
-			p, err := program.Load(data, program.Options{At: time.Now()})
-			if err != nil {
-				t.Fatal(err)
-			}
-			compiled, err := f.WithForms(p.Forms)
-			if err != nil {
-				t.Fatal(err)
-			}
-			for i, c := range f.Cases {
-				if !reflect.DeepEqual(c.Form, p.Forms[i]) {
-					t.Errorf("%s:%v: the program decodes to a different form", path, c.Pos)
-				}
+		for _, noShare := range []bool{false, true} {
+			name := filepath.Base(path)
+			if noShare {
+				name += "/no-share"
 			}
-			lisptest.Run(t, compiled, eval.NewTopLevel())
-		})
+			t.Run(name, func(t *testing.T) { testLispFile(t, path, noShare) })
+		}
+	}
+}
+
+func testLispFile(t *testing.T, path string, noShare bool) {
+	src, err := os.ReadFile(path)
+	if err != nil {
+		t.Fatal(err)
 	}
+	f, err := lisptest.Parse(path, string(src))
+	if err != nil {
+		t.Fatal(err)
+	}
+	data, err := lisp2gov.Compile(string(src), lisp2gov.Options{NoShare: noShare})
+	if err != nil {
+		t.Fatal(err)
+	}
+	p, err := program.Load(data, program.Options{At: time.Now()})
+	if err != nil {
+		t.Fatal(err)
+	}
+	compiled, err := f.WithForms(p.Forms)
+	if err != nil {
+		t.Fatal(err)
+	}
+	for i, c := range f.Cases {
+		if !reflect.DeepEqual(c.Form, p.Forms[i]) {
+			t.Errorf("%s:%v: the program decodes to a different form", path, c.Pos)
+		}
+	}
+	lisptest.Run(t, compiled, eval.NewTopLevel())
 }
diff --git a/program/program.go b/program/program.go
index c430330d50bffc30920bedbbf47f32680848a8a5..14a04fce3dbeb15d94f979c27790f5a0d221ce7c 100644
--- a/program/program.go
+++ b/program/program.go
@@ -106,6 +106,9 @@ // Locate names where an evaluation error happened: the OID of the
 // innermost list being evaluated and its depth, which is the cert that
 // holds it, plus the note of the innermost noted node around it. A .gov
 // program has no line numbers; this is its equivalent.
+//
+// A shared node has several parents but only one is recorded, the first
+// in program order, so the note around a shared node comes from there.
 func (p *Program) Locate(err error) string {
 	n := p.node(err)
 	if n == nil {