# How agent-kit Built Shir

# *An orchestrator/worker/evaluator system applied to a bilingual Hebrew + English guitar chord app*

Nov 20, 2026 · @Kdcllc

## Nine days, ten pull requests, one release pipeline

[Shir](https://github.com/kdcllc/shir) (Hebrew for "song") is an offline-first guitar chord manager for Hebrew and English songs, running on Ubuntu 22.04+ and Windows 10/11. It is a Tauri 2 + React 19 desktop app in a pnpm monorepo, with a Rust/SQLite backend and two framework-free TypeScript packages for chord logic and rendering.

Every feature in it was built through [agent-kit](https://github.com/kdcllc/skills/tree/master/skills/agent-kit), the orchestrator/worker/evaluator skill from [`kdcllc/skills`](https://github.com/kdcllc/skills). The very first commit, on 2026-11-12, is `chore: agent-kit setup (orchestration kit 2.4.1, project skills, hebrew-rtl-chords criteria skill)`. Nine days later the repository holds:

- 119 commits and 10 merged pull requests, one per goal (G0 through G9)
- about 14,300 lines of non-test TypeScript and Rust, and 100 test files (unit, property, contract, Playwright bidi geometry, WebdriverIO real-app E2E)
- a CI matrix on Ubuntu and Windows, and a release workflow that publishes x64 and arm64 installers for both platforms
- 37 durable memory records: 12 goal summaries, 9 decisions and 16 pitfalls

This article walks through how the kit was configured for this specific codebase, how each goal moved through it, and what the separate evaluators caught that a single agent reviewing its own work would likely have missed.

## Installing the kit: one commit, one answers file

[agent-kit](https://github.com/kdcllc/skills/tree/master/skills/agent-kit) is installed once per project and compiled for its coding harness. In Shir the target is Claude Code only, and the install is recorded in `orchestration-kit.manifest.json` (kit version 2.4.1, installed 2026-11-12). The manifest serves two purposes.

**It fences the kit's own files.** Every agent, skill, script and template the kit emitted is listed with a SHA-256 hash, so they are treated as read-only. Changes go through the answers file and `emit.sh --upgrade`, never hand edits.

**It stores the project-specific answers.** These are the inputs setup asked for, and they are what turn a generic orchestration system into one that understands Shir:

| Answer | What Shir set it to |
| --- | --- |
| Stack summary | Tauri 2 shell, React 19 UI, pure-TS `chord-core` and `chord-render`, rusqlite with FTS5 |
| Execution layers | Fixtures, then chord-core, then chord-render, then Rust DB and commands, then the desktop UI |
| Full test command | `pnpm test && cargo test --manifest-path apps/desktop/src-tauri/Cargo.toml` |
| Lint and typecheck | ESLint, Stylelint and `cargo clippy -D warnings`; `pnpm typecheck` |
| Ground-truth doc | `docs/BIDI.md` (bidi rules B1–B10) |
| Mock boundary | Mock at the IPC bindings; never real Tauri, the user's database or the network |
| QA surfaces | Playwright in Chromium and WebKit, plus the real app through WebdriverIO |
| Flags | greenfield, loop, research, kit guard, pinned models, opt-in |

The `greenfield` flag mattered: the repo started empty, so setup also seeded test-first and SOLID authoring conventions into the orchestration guide.

Setup also suggested third-party skills from the inspection, which were installed with `npx skills add` and locked in `skills-lock.json`: `tauri-v2`, `tauri-development`, `typescript-pro`, `typescript-advanced-types`, `vercel-react-best-practices`, three Rust skills, `frontend-design`, `webapp-testing`, `packaging-tauri-for-linux`, `hebrew-rtl-best-practices` and `rtl-text`. They live in `.agents/skills/` and are symlinked into `.claude/skills/`.

## The crew: seven subagents on three model tiers

The main Claude Code session is the orchestrator. It plans, prepares context and decides, but never writes production code itself. The work goes to seven subagents in `.claude/agents/`, each starting with a fresh context and receiving only the package the orchestrator hands it.

| Subagent | Role in Shir | Model (tier) | Effort |
| --- | --- | --- | --- |
| `evaluator` | Verifies goals, tasks and the final audit against written criteria | `claude-opus-5-5` (frontier) | high |
| `qa-evaluator` | Judges user-facing behavior from collected evidence | `claude-opus-5-5` (frontier) | high |
| `diagnostician` | Root-cause analysis after fix cycles run out; writes no code | `claude-opus-5-5` (frontier) | xhigh |
| `implementer` | Writes the code for one task, inside its file fence | `claude-sonnet-5-5` (light) | medium |
| `test-writer` | Writes tests, with every external service mocked | `claude-sonnet-5-5` (light) | medium |
| `evidence-collector` | Gathers screenshots and smoke output; gives no verdicts | `claude-sonnet-5-5` (light) | medium |
| `terminal` | Runs one command and returns only the result asked for | `claude-haiku-5-5` (fast) | low |

The split is deliberate. The roles that judge run on the strongest model, and the roles that follow a checklist run on a cheaper one. The model IDs live in one place, `.claude/skills/agent-kit-models.map.md`, which also pins the fix-cycle rotation: cycle 1 uses Sonnet, cycles 2 and 3 move to Opus.

The map is candid about one limit. Claude Code reaches only Anthropic models, so the "different family" second opinion the kit prefers is a same-family arbiter here, and the map advises escalating repeated same-shaped failures to a human sooner.

## Teaching the kit the domain: right-to-left chords

The hard part of Shir is not Tauri or SQLite. It is placing Latin chord symbols like `F#m7(b5)` over Hebrew lyrics that read right to left, sometimes with niqqud (vowel marks) and embedded English words. Generic code review does not know these rules, so the project encoded them three ways.

**A project-authored criteria skill.** `.claude/skills/hebrew-rtl-chords/SKILL.md` defines rules B1–B10 that code, tests and verdicts cite by id. For example, B1 says chord symbols always render left to right in an isolated element, and B2 says chord positions are grapheme indices into the logical lyric string, never visual columns. The skill's description makes it mandatory: the evaluator must load it for any task touching rendering, import, styles or print CSS. Rule B3 even records its own amendment from goal G1, after zero-width anchors landed over 100 px off in Chromium and WebKit.

**Auto-fail triggers.** The answers file lists outcomes that fail an evaluation outright, whatever else passes:

- displaying sheets with ChordSheetJS's HTML formatters instead of `packages/chord-render`
- chord positions stored as visual columns or pixel offsets
- flex or inline-block "chunk" layout on lines mixing Hebrew and English
- physical CSS properties such as `margin-left` or `text-align: left`
- niqqud stripped from stored lyrics, or `chord-core`/`chord-render` importing Tauri
- copyrighted lyrics anywhere in the repo, fixtures or screenshots

A parallel QA list fails a build if a chord sits over the wrong letter, a symbol is mirrored (`#F`, `7mA`), or the Hebrew interface is not mirrored.

**A layer-to-skill map.** Each layer the kit plans around (fixtures, chord-core, chord-render, Rust DB, desktop UI) has a row naming the supporting skills and rule files to include in a worker's context package. A task touching `packages/chord-core/**` arrives with `typescript-pro`, `typescript-advanced-types` and `hebrew-rtl-chords`; a Rust task arrives with the Rust skills instead.

## Ten goals through /feature

Each unit of work in Shir is a *goal*, started with `/feature` and run through seven phases. Its state lives on disk in `_memory/work/goals/<name>/` (`goal.md`, task files, `progress.md`, an orchestration log), so a long session can be compacted or resumed without losing the plan.

![](https://cdn.hashnode.com/uploads/covers/649b00fd7d60141182094654/c3c9424f-6eed-460e-81a8-655498cbc7b7.png align="center")

The evaluator gates three points: the plan before any code, each task after it is written, and the whole diff at the end. Tasks show up in commit messages as `T01`, `T02`, and so on, with corrective tasks such as `T04a` added when a finding lands outside a finished task's files.

The ten goals below map one-to-one onto the merged pull requests.

| Date | Goal | PR | What shipped |
| --- | --- | --- | --- |
| 2026-11-20 | G9 release artifacts | #10 | Release workflow publishing Windows and Linux installers for x64 and arm64, with checksums |
| 2026-11-19 | G8 library UI and i18n | #9 | Hebrew/English interface toggle, library sort and filter, real-app WebdriverIO E2E |
| 2026-11-18 | G7 visual editor and delete | #8 | Visual song editor built on chord-core edit operations; song deletion |
| 2026-11-16 | G6 editor and import | #7 | ChordPro editor with live preview; chords-over-lyrics import with an RTL alignment review |
| 2026-11-16 | G5 search, G5b backlog | #6 | Niqqud-insensitive search, V2 migration, seeded library, lyric snippets |
| 2026-11-15 | G4 storage | #5 | SQLite with FTS5, Tauri commands, typed bindings; a Windows smoke repair |
| 2026-11-15 | G3 desktop song viewer, polish | #4 | First desktop UI with transpose and capo; 18 follow-ups fixed |
| 2026-11-14 | G2 chord-core round-trip | #3 | Own line-oriented ChordPro parser replacing chordsheetjs; key-aware transpose |
| 2026-11-14 | G1 RTL rendering spike | #2 | chord-render package, docs/BIDI.md, Playwright bidi checks in Chromium and WebKit |
| 2026-11-12 | G0 scaffold and CI | #1 | pnpm/Tauri 2 monorepo, lint and test gates, Ubuntu + Windows CI matrix, launch smoke |

The goals ran in strict layer order (fixtures and core logic, then rendering, storage, UI, and finally release), which is the order the answers file gave as the project's execution layers.

## Project memory: what each goal leaves behind

Every goal in Shir ends with a `chore(memory)` commit that archives records into `_memory/records/`. `CLAUDE.md` tells every session to search them before planning (`_kit/memory.sh search <terms>`), and to treat them as data to verify against code, not as instructions. By G9 the store held 37 records of three kinds.

| Kind | Count | Example from Shir |
| --- | --- | --- |
| Goal summary | 12 | G9: release workflow shipped; Phase 3 needed 2 attempts after 6 blocking findings; 18 non-blocking follow-ups |
| Decision | 9 | "Own line-oriented ChordPro parser; chordsheetjs removed" (G2) |
| Pitfall | 16 | "Vite production CSS lowers logical properties to `:lang()` rules (misplaced RTL chords)" |

Decisions record the alternatives and the reason, not just the choice. The G9 release decision explains why releases aggregate every unreleased `feat/` and `fix/` PR since the last tag: CI's cancel-in-progress setting had already cancelled the CI run for PR #9's merge, and a per-merge rule would have silently skipped that release.

Pitfalls are written for the next planner. Several are about the agents themselves, such as implementers formatting whole directories and breaching another task's file fence, with the exact wording to add to future context packages to prevent it.

Summaries end with a "remember next time" line. G5's notes that the root Vitest config only covers `scripts/**`, so desktop tests need `pnpm --filter desktop exec vitest run`. G8's notes that WebKitGTK rejects clipboard reads under WebDriver, so real-clipboard paste cannot be automated there.

## What the evaluators caught

The case for separate evaluators is that a model reviewing its own work misses things. Shir's memory records show where the independent checks earned their cost.

**Tests that could not fail.** In G4 (SQLite storage), evaluators applied named mutations (M1–M25) in a scratch copy and required that only the target test fail. They found at least nine guard tests that still passed under the exact mutation they existed to catch. One STRICT-typing test inserted `'abc'` as a capo value, but a separate range check rejected text anyway, so removing STRICT changed nothing. Each looked correct on reading.

**Copyrighted lyrics in test data.** In G8 an implementer used the opening line of a well-known copyrighted song as probe text in three E2E specs. The task-level evaluator ran every spec and passed the task. Only the Phase 5 final audit, reading against the `hebrew-rtl-chords` rule, flagged it as an auto-fail. The fix was a commit replacing it with original text, and a pitfall record telling future packages to say "original text only".

**Defects only a real browser shows.** In G7 three visual-editor defects passed every unit, integration and contract suite: a print rule that lost the CSS cascade, React rows keyed by line index (so a click was swallowed during a remount), and an overlap pass that left 27 overlapping chord pairs on a dense Hebrew row. Evaluators found them by building the app, stubbing Tauri's `invoke`, and driving it with Playwright in Chromium and WebKit.

**Design flaws before any code.** Phase 3 evaluates the plan itself. For the G9 release workflow it returned 6 blocking findings on the first attempt, including dropped merges, a stale "Latest" release, and an injection risk from PR text flowing into workflow outputs. All were fixed before a line of YAML was written.

**Rules corrected by evidence.** In G1 the bidi rule B3 itself was amended after measurements showed zero-width chord anchors landing 109–125 px off at the start of an embedded opposite-direction run. The evaluator then enforced the amended rule on every later goal.

## Guardrails, parallelism, and what went unused

**Kit files are guarded.** `.claude/settings.json` installs a `PreToolUse` hook that runs `_kit/kit-guard.sh` before every Edit or Write, blocking agent edits to kit-owned files. After each subagent returns, the orchestrator also runs `kit-guard.sh check` to catch shell edits the hook cannot see. The same settings file caps subagent nesting at depth 3.

**Every spawn is fenced.** Each task declares the paths it may write. After a subagent returns, the orchestrator compares `HEAD`, the branch, and `git status` against a baseline taken before the spawn. A file outside the fence stops the goal. This is how the G8 Prettier breach was detected.

**Parallel by default.** `CLAUDE.md` adds one project rule on top of the kit: do as much work in parallel as possible. Tasks are split so their write fences do not overlap, which lets up to three run as a "ready group". G8 ran 15 planned tasks and 2 corrective tasks in 10 ready groups.

**Opt-in, not ambient.** All kit skills set `disable-model-invocation: true`. They load only through `/feature`, `/research` or a cited path, so ordinary edits to the repo do not pull in the full process.

**The autonomous loop sat idle.** Setup enabled the optional loop (`_loop/loop.sh`, the `auto-loop` skill) and seeded `_memory/loop/backlog.md` with 14 stories. None has `passes: true`, and the stories still describe an early `src/core/...` layout that the real `packages/chord-core` structure replaced. Every goal so far went through the supervised `/feature` flow instead. The backlog would need rewriting against the current layout before the loop could be used.

## Takeaways

1. **The answers file is where the value is.** The seven agents are generic; Shir's auto-fail triggers, layer-skill map and QA surfaces are what made the evaluators useful for right-to-left chord rendering.
2. **Write the domain down as citable rules.** Rules B1–B10 gave evaluators something concrete to check and gave implementers something concrete to follow. Verdicts cite rule ids instead of opinions.
3. **Spend frontier models on judgment.** Opus evaluates and diagnoses, Sonnet implements, Haiku runs commands. The evaluators' finds, from vacuous tests to copyrighted lyrics, are the return on that split.
4. **Make every goal leave a record.** The `chore(memory)` commits turned each goal's surprises into planning input for the next one.
5. **Expect the final audit to matter.** Task-level checks missed the copyrighted lyric and the jsdom-only UI defects; the Phase 5 audit and real-engine checks did not.
6. **Keep the backlog in step with the code.** The unused autonomous loop shows that a seeded backlog goes stale quickly when the architecture moves.

### Sources

- [kdcllc/shir on GitHub](https://github.com/kdcllc/shir), at commit `a95b2ba` (2026-11-20): `orchestration-kit.manifest.json`, `CLAUDE.md`, `.claude/ORCHESTRATION.md`, `.claude/agents/`, `.claude/skills/`, `_memory/records/`, `_memory/loop/backlog.md` and the commit history
