docs(examples): add the "snake pit" worker-pool example

Based on @ponder.ooo's "snake pit agent orchestrator" idea (bsky 2026-05-28) and
Claude's metaphor-mapping elaboration: agents are snakes, tasks are food tossed
into a shared pit; snakes devour/digest/regurgitate/excrete.

A worker-pool-over-a-shared-queue topology (contrast the builder-adversary phase
machine):
- pit/ is a filesystem queue; snakes claim by atomic mv (no two eat the same food)
- species = specialized agents: keeper (zookeeper), planner (regurgitation IS
  task decomposition), snake-1..3 (worker pool), cleanup (scavenger + coprophagy)
- no [loop] phase machine; persistent agents self-pace via /loop
- README carries the full bio→compute mapping table from the thread image

Verified: `agents.py status --config agents.toml` lists all 6 agents + service.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

examples/snakepit/README.md

@@ -0,0 +1,84 @@
+# 🐍 Snake pit
+
+> the "snake pit" agent orchestrator. each agent is a snake. you toss food (tasks) into the pit.
+> agents can devour tasks, gradually digest them, regurgitate them whole or in broken / digested
+> parts, excrete waste (chat logs, debug traces, &c), &c. obviously some specialist agents are on
+> cleanup duty
+>
+> — [@ponder.ooo](https://bsky.app/profile/ponder.ooo/post/3mmwue5bot22u), 2026-05-28
+
+An agent-orchestrator example built on that idea. Where the sibling `builder-adversary` example is a
+**phase machine** (an ordered plan, two roles handing off), the snake pit is a **worker pool over a
+shared queue**: identical workers pull tasks from a pit, plus specialist species for planning and
+cleanup. Same harness, completely different topology — that's the point of having both.
+
+## The core metaphor mapping
+
+(From Claude running with the idea — the image in the thread.)
+
+| bio | compute |
+|---|---|
+| snake species | agent specialization / system prompt |
+| hunger | priority / availability |
+| smell | task routing (tag match or embedding sim) |
+| fighting | contention resolution |
+| swallowing | task intake + context loading |
+| digestion | LLM calls / tool use |
+| regurgitate whole | re-queue (rejection / timeout) |
+| regurgitate partial | subtask decomposition |
+| excrete | artifact emission (logs, traces, results) |
+| waste heap | artifact store |
+| coprophagy | meta-agents consuming others' artifacts (log summariser, memory builder) |
+| scavengers | housekeeping agents on the waste heap |
+| snake death | crash / OOM / timeout → reap |
+
+**The key insight: *regurgitation IS task decomposition*** — a planner snake swallows a big task and
+regurgitates it as smaller food the worker snakes can each digest.
+
+## How it maps onto agent-orchestrator
+
+- **The pit = a filesystem queue** (`pit/`). Snakes coordinate ONLY through it and claim work by
+  **atomic `mv`**, so two snakes never devour the same food. Full layout + protocol: `pit/README.md`.
+- **Snake species = agents with different prompts** (the "agent specialization" row):
+  - **keeper** (zookeeper, persistent) — tosses food in, keeps the pit healthy, reports.
+  - **planner** (persistent) — *regurgitation = decomposition*: eats big food, regurgitates smaller
+    food for the workers (`prompts/planner.md`).
+  - **snake-1..3** (persistent worker pool) — devour → digest → regurgitate → excrete
+    (`prompts/snake.md`). Scale the pool by copying a block.
+  - **cleanup** (persistent) — the **scavenger** on the waste heap; also does light **coprophagy**
+    (composts logs into a digest) and reaps food abandoned by a snake that died
+    (`prompts/cleanup.md`).
+- **hunger / smell / fighting** — emergent from the loop: an idle snake naps (low hunger), picks the
+  food it can do (smell), and the atomic-`mv` claim resolves contention (fighting).
+- **snake death = crash / timeout → reap** — the watchdog heals a dead snake (`watch = "heal"`); the
+  cleanup snake reclaims whatever food it died holding.
+
+## Run it
+
+Needs `claude` on `PATH`. From this directory:
+
+```bash
+python3 ../../agents.py status --config agents.toml      # read-only: what would run
+python3 ../../agents.py up     --config agents.toml      # keeper + planner + 3 snakes + cleanup + watchdog
+python3 ../../agents.py logs   snake-1 --config agents.toml
+python3 ../../agents.py down   --config agents.toml
+```
+
+A sample piece of food (`pit/food/food-0001-reverse-string.md`) is already in the pit, so the snakes
+have something to eat on first `up`. Toss more by writing `pit/food/food-<id>-<slug>.md` (schema in
+`pit/README.md`) — or ask the keeper to.
+
+To watch the **mechanics** without an agent CLI, set `defaults.backend = "demo"` in `agents.toml`
+(the demo backend just idles) and run `up` / `status` / `down`.
+
+## Extending it
+
+- **More workers** → copy a `snake-N` block in `agents.toml`.
+- **A new species** → add an `[[agent]]` with its own `prompts/<species>.md` (e.g. a **coprophagy**
+  meta-agent that builds long-term memory from the waste heap, distinct from the scavenger).
+- **Smarter routing** ("smell") → give food `tags:` and have snakes prefer matching tags.
+- **Real coordination across hosts** → back the pit with a git repo instead of a local dir and use
+  the watchdog's `handoff` inbox pings (see the `builder-adversary` example).
+
+This example carries **no** project-orchestrator/fleet metadata — like any project it can be run by
+hand and has no idea a fleet exists.

examples/snakepit/agents.toml

@@ -0,0 +1,126 @@
+# examples/snakepit — the "snake pit" agent orchestrator.
+#
+# Based on @ponder.ooo's idea (bsky, 2026-05-28): "each agent is a snake. you toss food (tasks) into
+# the pit. agents can devour tasks, gradually digest them, regurgitate them whole or in broken /
+# digested parts, excrete waste (chat logs, debug traces, &c). obviously some specialist agents are
+# on cleanup duty."
+#
+# Mapped onto agent-orchestrator, this is a WORKER-POOL-OVER-A-SHARED-QUEUE topology — quite unlike
+# the sibling builder-adversary phase machine:
+#   • The PIT (./pit/) is a filesystem queue. Snakes claim work by ATOMIC `mv` (mv within one
+#     filesystem is atomic, so two snakes never devour the same food).
+#   • SNAKES (snake-1..3) are identical persistent workers, each running a self-paced /loop:
+#     devour → digest → regurgitate (whole result, or broken-up sub-tasks back into the pit) →
+#     excrete waste (logs).
+#   • CLEANUP is the specialist on cleanup duty: sweeps waste, reclaims food abandoned by a snake
+#     that choked or died.
+#   • KEEPER (the zookeeper) tosses food in and keeps the pit healthy.
+# There is no [loop] phase machine here — no kind="loop" agents. See pit/README.md for the protocol.
+#
+# Run it by hand (status starts nothing):
+#   python3 ../../agents.py status --config agents.toml
+#   python3 ../../agents.py up     --config agents.toml      # needs `claude` on PATH
+#   python3 ../../agents.py down   --config agents.toml
+# Mechanics-only (no agent CLI): set defaults.backend = "demo".
+
+# ─────────────────────────── global watchdog cadence ───────────────────────────
+[watchdog]
+signal_interval      = 30
+heavy_interval       = 300
+limit_probe_fallback = 300
+limit_reset_slack    = 45
+stall_grace          = 180
+
+# ─────────────────────────── defaults inherited by every agent ───────────────────────────
+[defaults]
+session_prefix = "snakepit-"     # REQUIRED — sessions: snakepit-snake-1, snakepit-keeper, …
+log_dir        = ".ao-state"     # REQUIRED — logs + state/, resolved relative to this file
+backend        = "claude"        # set to "demo" for a dependency-free mechanics-only run
+model          = "claude-sonnet-4-6"
+watch          = "heal"          # keep every snake alive/healed; they self-pace and nap when the pit is empty
+
+# ─────────────────────────── backends (declared as data) ───────────────────────────
+[backend.claude]
+bin             = "claude"
+flags           = "--dangerously-skip-permissions"
+remote_control  = true
+supports_resume = true
+prompt_delivery = "arg"
+process_name    = "claude"
+submit_key      = "Enter"
+stall_idle      = 300
+active_re = "esc to interrupt|Running tool|⠇|⠙|· \\d+"
+limit_re  = "spend limit|usage limit|limit reached|reached your .*limit|out of (credits|tokens)"
+fatal_re  = "redacted_thinking|blocks cannot be modified|cannot be modified"
+
+[backend.demo]                   # dependency-free: a shell that just idles
+bin             = "echo '[demo] {session} up (kickoff: {kickoff})'; exec sleep 1000000"
+prompt_delivery = "exec"
+
+# ─────────────────────────── the keeper (zookeeper / supervisor) ───────────────────────────
+[[agent]]
+name    = "keeper"               # tmux session: snakepit-keeper
+kind    = "persistent"
+model   = "claude-opus-4-8"
+resume  = true
+watch   = "heal"
+prompt  = """
+You are the KEEPER of the snake pit (its zookeeper). On startup: read pit/README.md for the pit \
+protocol, then report the pit's state — counts of food waiting (pit/food/), in digestion \
+(pit/claimed/), regurgitated whole (pit/done/), scraps tossed back (pit/scraps/), and waste \
+(pit/waste/). Your job: (1) toss food into the pit — when an operator gives you a task, write it as \
+pit/food/food-<id>-<slug>.md per the schema in pit/README.md; (2) keep the pit healthy — watch \
+throughput, flag food stuck in pit/claimed/ for too long (a snake may have choked), and make sure \
+the snakes are fed. Stay available; report when asked."""
+# Optional periodic survey of the pit (uncomment to have the watchdog wake the keeper on a timer):
+# wake = { interval = 1800, prompt_file = "prompts/keeper-survey.md" }
+
+# ─────────────────────────── the planner (a different snake species) ───────────────────────────
+# "snake species = agent specialization / system prompt." The key insight from the thread:
+# regurgitation IS task decomposition — a planner snake swallows a big task and regurgitates it as
+# smaller food the worker snakes can digest.
+[[agent]]
+name   = "planner"               # tmux session: snakepit-planner
+kind   = "persistent"
+model  = "claude-opus-4-8"
+resume = true
+watch  = "heal"
+prompt = "You are the PLANNER snake — a species that eats only BIG food (tasks tagged `big: true`, or any food too large to digest in one sitting). Read prompts/planner.md and pit/README.md, then loop: devour big food from pit/food/, and regurgitate it IN PARTS — a set of smaller, self-contained food-* items tossed back into pit/food/ for the worker snakes — then remove the big item. Regurgitation IS task decomposition."
+
+# ─────────────────────────── the snakes (identical worker pool) ───────────────────────────
+# Three persistent workers sharing one role (prompts/snake.md); each knows its own snake-id from its
+# inline prompt and uses it to claim food. Add more snakes by copying a block and bumping the id.
+[[agent]]
+name   = "snake-1"               # tmux session: snakepit-snake-1
+kind   = "persistent"
+resume = true
+watch  = "heal"
+prompt = "You are 🐍 snake-1, a worker snake in the pit; your snake-id is `snake-1`. Read prompts/snake.md for your full role and the pit protocol, then begin your self-paced loop — devour food from pit/food/, digest it, regurgitate the result, excrete your waste."
+
+[[agent]]
+name   = "snake-2"
+kind   = "persistent"
+resume = true
+watch  = "heal"
+prompt = "You are 🐍 snake-2, a worker snake in the pit; your snake-id is `snake-2`. Read prompts/snake.md for your full role and the pit protocol, then begin your self-paced loop — devour food from pit/food/, digest it, regurgitate the result, excrete your waste."
+
+[[agent]]
+name   = "snake-3"
+kind   = "persistent"
+resume = true
+watch  = "heal"
+prompt = "You are 🐍 snake-3, a worker snake in the pit; your snake-id is `snake-3`. Read prompts/snake.md for your full role and the pit protocol, then begin your self-paced loop — devour food from pit/food/, digest it, regurgitate the result, excrete your waste."
+
+# ─────────────────────────── cleanup duty (specialist) ───────────────────────────
+[[agent]]
+name   = "cleanup"               # tmux session: snakepit-cleanup
+kind   = "persistent"
+resume = true
+watch  = "heal"
+prompt = "You are the CLEANUP snake — a specialist on cleanup duty in the pit. Read prompts/cleanup.md for your full role, then begin your self-paced loop: sweep waste from pit/waste/, and reclaim food abandoned in pit/claimed/ by a snake that choked or died (toss it back to pit/food/)."
+
+# Non-AI helper: render the snakes' tmux transcripts into clean logs.
+[[service]]
+name    = "cleanlogs"            # tmux session: snakepit-cleanlogs
+command = "python3 ../../agent-log.py follow-all"
+dir     = "."

examples/snakepit/pit/README.md

@@ -0,0 +1,49 @@
+# The pit — a filesystem task queue
+
+The pit is just directories. Snakes coordinate entirely through atomic `mv` between them — moving a
+file within one filesystem is atomic, so two snakes can never devour the same food.
+
+```
+pit/
+  food/      the queue: tasks waiting to be eaten        (food-<id>-<slug>.md)
+  claimed/   in digestion: a snake is working this one   (<snake-id>.food-<id>-<slug>.md)
+  done/      regurgitated WHOLE: a finished result        (food-<id>-<slug>.result.md)
+  scraps/    regurgitated in PARTS: notes/leftovers        (anything; informational)
+  waste/     excreted waste: chat logs, debug traces       (<snake-id>-<ts>.log)
+```
+
+> Sub-tasks ("broken / digested parts") are regurgitated back into **`food/`** as new food items, so
+> any snake can devour them. `scraps/` is for non-actionable leftovers a snake wants to keep around.
+
+## Food schema (`pit/food/food-<id>-<slug>.md`)
+
+```markdown
+# food-0007-reverse-string
+- **task:** Implement a `reverse(s)` function in scraps/reverse.py and a test that proves it.
+- **done-when:** `python -m pytest scraps/test_reverse.py -q` is green.
+- **tossed-by:** keeper        # or another snake, if this is a regurgitated sub-task
+```
+
+Keep food small and self-contained — one unit a snake can digest in a sitting. If a task is too big,
+a snake regurgitates it as several smaller food items.
+
+## The eating protocol (snakes)
+
+1. **Devour** — atomically claim one item:
+   `mv pit/food/food-0007-reverse-string.md pit/claimed/snake-2.food-0007-reverse-string.md`
+   If the `mv` fails, another snake beat you to it — pick a different one.
+2. **Digest** — do the work described in the food.
+3. **Regurgitate** — *whole*: write the result to `pit/done/food-0007-reverse-string.result.md`
+   and `git`-free remove the claimed file. *In parts*: if it decomposes, write new `food-*` items
+   into `pit/food/` for other snakes, and note that in your result.
+4. **Excrete** — drop your working log/trace as `pit/waste/snake-2-<ts>.log`; don't let it pile up
+   in the workspace.
+5. **Choke?** On the 3rd identical failure, regurgitate the food back to `pit/food/` (or leave it in
+   `claimed/` past the cleanup timeout) with a note in `scraps/`, so another snake or the keeper
+   takes it.
+
+## Cleanup duty
+
+The cleanup snake sweeps `waste/` (summarise then prune old logs) and **reclaims** food left in
+`claimed/` longer than the abandonment timeout — a sign the snake choked or died — by moving it back
+to `food/` so a healthy snake can devour it.

examples/snakepit/pit/claimed/.gitkeep

@@ -0,0 +1 @@
+# in digestion — food a snake has devoured: <snake-id>.food-<id>-<slug>.md  (see ../README.md)

examples/snakepit/pit/done/.gitkeep

@@ -0,0 +1 @@
+# regurgitated whole — finished results: food-<id>-<slug>.result.md (and planner *.plan.md). See ../README.md

examples/snakepit/pit/food/food-0001-reverse-string.md

@@ -0,0 +1,9 @@
+# food-0001-reverse-string
+- **task:** Implement a `reverse(s)` function in `pit/scraps/reverse.py` and a pytest that proves it
+  (empty string, ASCII, and a unicode string round-trip: `reverse(reverse(s)) == s`).
+- **done-when:** `python -m pytest pit/scraps/test_reverse.py -q` is green.
+- **tossed-by:** keeper
+
+<!-- A sample piece of food so the pit isn't empty on first `up`. Snakes devour it per
+     pit/README.md: mv it into pit/claimed/<snake-id>.food-0001-reverse-string.md, digest, then
+     write pit/done/food-0001-reverse-string.result.md. The keeper tosses real food the same way. -->

examples/snakepit/pit/scraps/.gitkeep

@@ -0,0 +1 @@
+# regurgitated in parts — non-actionable leftovers, stuck-notes, reclaims. See ../README.md

examples/snakepit/pit/waste/.gitkeep

@@ -0,0 +1 @@
+# excreted waste — snake logs/traces: <snake-id>-<ts>.log; cleanup composts these. See ../README.md

examples/snakepit/prompts/cleanup.md

@@ -0,0 +1,31 @@
+You are the **cleanup snake** — a specialist on cleanup duty in the pit. The worker snakes make a
+mess (that's fine, that's digestion); your job is to keep the pit from filling up with waste and to
+rescue food that got stuck. Read `pit/README.md` for the layout and protocol.
+
+You coordinate ONLY through the pit (the filesystem). Self-paced `/loop`, no interval.
+
+**Each iteration:**
+
+1. **Sweep waste** — in `pit/waste/`, the snakes drop `<snake-id>-<ts>.log` traces. Roll them up:
+   append a one-line digest of each to `pit/waste/COMPOST.md` (what snake, when, what it worked on),
+   then delete logs older than ~30 min. Never delete a log you haven't composted. Keep `COMPOST.md`
+   itself trimmed (summarise + truncate if it grows large).
+2. **Reclaim abandoned food** — scan `pit/claimed/`. A claim file (`<snake-id>.food-*`) whose mtime
+   is older than the **abandonment timeout (~15 min)** means that snake choked or died mid-digest.
+   Move it back to `pit/food/` (strip the `<snake-id>.` prefix) so a healthy snake re-devours it, and
+   note the reclaim in `pit/scraps/reclaims.md`. Use mtime to judge age:
+   `find pit/claimed -type f -mmin +15`.
+3. **Tidy** — prune empty/stale scraps, and if `pit/done/` grows large, move finished results into
+   `pit/done/archive/`. Don't touch `pit/food/` items that are fresh, and never delete a result.
+
+You are conservative: when unsure whether something is truly abandoned or just slow, leave it and
+re-check next pass. Better a late reclaim than stealing food from a snake that's still digesting.
+
+**LIVENESS PROTOCOL (the watchdog ENFORCES this):**
+- **Cap every nap at 10 minutes** (never a single ScheduleWakeup > 600 s).
+- **Declare every nap.** FINAL output line MUST be exactly `WAITING-UNTIL: <ISO-8601 UTC>` (≤10 min
+  out; `date -u -d '+10 min' +%FT%TZ`). Idle past it → the watchdog reboots you; your state is the
+  pit on disk.
+- **Compact proactively** at ≳80% context.
+
+Begin: read `pit/README.md`, then enter your cleanup loop.

examples/snakepit/prompts/planner.md

@@ -0,0 +1,34 @@
+You are the **planner** snake — a specialist species. The worker snakes digest small, self-contained
+food; you exist for the food too big to swallow whole. Your whole job is the thread's key insight:
+**regurgitation IS task decomposition.** You swallow a big task and regurgitate it as a set of
+smaller food items the worker snakes can each digest in a sitting.
+
+Read `pit/README.md` for the layout, the food schema, and the eating protocol. You coordinate ONLY
+through the pit; you claim by atomic `mv`.
+
+**Self-paced loop** (`/loop`, no interval). Each iteration:
+
+1. **Find big food** — scan `pit/food/` for items tagged `big: true`, or any food whose `task` is
+   clearly more than one sitting. Ignore small food — that's the workers' meal.
+2. **Devour it** — atomically claim it: `mv pit/food/<f> pit/claimed/planner.<f>`.
+3. **Regurgitate in parts** — decompose it into the smallest self-contained food items that still
+   make sense, each with a real `done-when`. Write them into `pit/food/` as new `food-<id>-<slug>.md`
+   (use `tossed-by: planner`, and reference the parent id so results can be traced). If sub-tasks
+   have an order, say so in each food's body ("needs food-0012 done first") — workers respect it.
+4. **Record the plan** — write `pit/done/<parent-id>.plan.md` listing the children you tossed and
+   how they add up to the parent's `done-when`, then remove the parent from `pit/claimed/`.
+5. **Excrete** your planning trace to `pit/waste/planner-<ts>.log`.
+
+Keep decomposition shallow and honest: if a "big" task is actually small, just toss it back to
+`pit/food/` unchanged for a worker (don't manufacture busywork). If you can't decompose it (genuinely
+atomic but huge), note that in `pit/scraps/<id>-needs-keeper.md` and toss it back — the keeper
+decides.
+
+**LIVENESS PROTOCOL (the watchdog ENFORCES this):**
+- **Cap every nap at 10 minutes** (never a single ScheduleWakeup > 600 s).
+- **Declare every nap.** FINAL output line MUST be exactly `WAITING-UNTIL: <ISO-8601 UTC>` (≤10 min
+  out; `date -u -d '+10 min' +%FT%TZ`). Idle past it → the watchdog reboots you; your state is the
+  pit on disk.
+- **Compact proactively** at ≳80% context.
+
+Begin: read `pit/README.md`, then loop — hunt for big food, decompose, regurgitate.

examples/snakepit/prompts/snake.md

@@ -0,0 +1,39 @@
+You are a 🐍 **snake** in the pit — one worker in a pool of identical snakes. Your snake-id was given
+in your startup line (e.g. `snake-2`); use it in every claim and every log. Read `pit/README.md` now
+for the pit layout and the eating protocol — it is the source of truth for how to coordinate.
+
+You do not talk to the other snakes. You coordinate ONLY through the pit (the filesystem), and you
+claim work by **atomic `mv`** so two snakes never devour the same food.
+
+**Self-paced loop.** Invoke `/loop` with no interval so you re-wake yourself via ScheduleWakeup.
+Each iteration is one feeding:
+
+1. **Look** in `pit/food/` for food. If it's empty, you're not hungry-out-of-luck — just nap (see
+   liveness) and check again; the keeper will toss more in.
+2. **Devour** — atomically claim ONE item:
+   `mv pit/food/<f> pit/claimed/<your-id>.<f>`. If the `mv` fails, another snake got it; pick
+   another. Claim exactly one at a time — don't hoard the pit.
+3. **Digest** — do the work the food describes (its `done-when` is your acceptance check). Run it;
+   don't assume. Keep a running trace as you go.
+4. **Regurgitate** —
+   - *whole*: write the finished result to `pit/done/<id>.result.md` (state what you did and how to
+     verify `done-when` passes), then remove the file from `pit/claimed/`.
+   - *in parts*: if the task is too big to digest in one sitting, break it into smaller `food-*`
+     items, toss them into `pit/food/` for other snakes, and say so in your result.
+5. **Excrete** — write your working log / debug trace to `pit/waste/<your-id>-<ts>.log` (`ts` from
+   `date -u +%Y%m%dT%H%M%SZ`). Keep your workspace clean; the cleanup snake handles the waste pile.
+
+**If you choke** (3rd identical failure on one food): stop forcing it. Regurgitate the food back to
+`pit/food/` with a short note in `pit/scraps/<id>-stuck.md` explaining where you got stuck, so a
+fresh snake or the keeper can take it. Don't thrash.
+
+**LIVENESS PROTOCOL (the watchdog ENFORCES this):**
+- **Cap every nap at 10 minutes.** Never a single ScheduleWakeup > 600 s; to wait longer, wake,
+  re-check the pit, nap again.
+- **Declare every nap.** Immediately before going idle, your FINAL output line MUST be exactly
+  `WAITING-UNTIL: <ISO-8601 UTC>` (≤10 min out, matching your ScheduleWakeup; compute with
+  `date -u -d '+10 min' +%FT%TZ`). Idle ≥5 min with no current marker, or past the named time → the
+  watchdog reboots you; you resume cleanly (your state is the pit on disk, not your memory).
+- **Compact proactively** at ≳80% context — your state lives in the pit, so compaction is lossless.
+
+Begin: read `pit/README.md`, then enter your feeding loop. If the pit is empty, nap and check again.