How agent-kit Built Shir
See how agent-kit’s orchestrator, worker and evaluator agents built Shir, a Hebrew-English guitar chord app, in 9 days and 10 PRs with Claude Code.

An orchestrator/worker/evaluator system applied to a bilingual Hebrew + English guitar chord app
Nov 20, 2026 · @Kdcllc
Nine days, ten pull requests, one release pipeline
Shir (Hebrew for "song") is an offline-first guitar chord manager for Hebrew and English songs, running on Ubuntu 22.04+ and Windows 10/11. It is a Tauri 2 + React 19 desktop app in a pnpm monorepo, with a Rust/SQLite backend and two framework-free TypeScript packages for chord logic and rendering.
Every feature in it was built through agent-kit, the orchestrator/worker/evaluator skill from kdcllc/skills. The very first commit, on 2026-11-12, is chore: agent-kit setup (orchestration kit 2.4.1, project skills, hebrew-rtl-chords criteria skill). Nine days later the repository holds:
- 119 commits and 10 merged pull requests, one per goal (G0 through G9)
- about 14,300 lines of non-test TypeScript and Rust, and 100 test files (unit, property, contract, Playwright bidi geometry, WebdriverIO real-app E2E)
- a CI matrix on Ubuntu and Windows, and a release workflow that publishes x64 and arm64 installers for both platforms
- 37 durable memory records: 12 goal summaries, 9 decisions and 16 pitfalls
This article walks through how the kit was configured for this specific codebase, how each goal moved through it, and what the separate evaluators caught that a single agent reviewing its own work would likely have missed.
Installing the kit: one commit, one answers file
agent-kit is installed once per project and compiled for its coding harness. In Shir the target is Claude Code only, and the install is recorded in orchestration-kit.manifest.json (kit version 2.4.1, installed 2026-11-12). The manifest serves two purposes.
It fences the kit's own files. Every agent, skill, script and template the kit emitted is listed with a SHA-256 hash, so they are treated as read-only. Changes go through the answers file and emit.sh --upgrade, never hand edits.
It stores the project-specific answers. These are the inputs setup asked for, and they are what turn a generic orchestration system into one that understands Shir:
| Answer | What Shir set it to |
|---|---|
| Stack summary | Tauri 2 shell, React 19 UI, pure-TS chord-core and chord-render, rusqlite with FTS5 |
| Execution layers | Fixtures, then chord-core, then chord-render, then Rust DB and commands, then the desktop UI |
| Full test command | pnpm test && cargo test --manifest-path apps/desktop/src-tauri/Cargo.toml |
| Lint and typecheck | ESLint, Stylelint and cargo clippy -D warnings; pnpm typecheck |
| Ground-truth doc | docs/BIDI.md (bidi rules B1–B10) |
| Mock boundary | Mock at the IPC bindings; never real Tauri, the user's database or the network |
| QA surfaces | Playwright in Chromium and WebKit, plus the real app through WebdriverIO |
| Flags | greenfield, loop, research, kit guard, pinned models, opt-in |
The greenfield flag mattered: the repo started empty, so setup also seeded test-first and SOLID authoring conventions into the orchestration guide.
Setup also suggested third-party skills from the inspection, which were installed with npx skills add and locked in skills-lock.json: tauri-v2, tauri-development, typescript-pro, typescript-advanced-types, vercel-react-best-practices, three Rust skills, frontend-design, webapp-testing, packaging-tauri-for-linux, hebrew-rtl-best-practices and rtl-text. They live in .agents/skills/ and are symlinked into .claude/skills/.
The crew: seven subagents on three model tiers
The main Claude Code session is the orchestrator. It plans, prepares context and decides, but never writes production code itself. The work goes to seven subagents in .claude/agents/, each starting with a fresh context and receiving only the package the orchestrator hands it.
| Subagent | Role in Shir | Model (tier) | Effort |
|---|---|---|---|
evaluator |
Verifies goals, tasks and the final audit against written criteria | claude-opus-5-5 (frontier) |
high |
qa-evaluator |
Judges user-facing behavior from collected evidence | claude-opus-5-5 (frontier) |
high |
diagnostician |
Root-cause analysis after fix cycles run out; writes no code | claude-opus-5-5 (frontier) |
xhigh |
implementer |
Writes the code for one task, inside its file fence | claude-sonnet-5-5 (light) |
medium |
test-writer |
Writes tests, with every external service mocked | claude-sonnet-5-5 (light) |
medium |
evidence-collector |
Gathers screenshots and smoke output; gives no verdicts | claude-sonnet-5-5 (light) |
medium |
terminal |
Runs one command and returns only the result asked for | claude-haiku-5-5 (fast) |
low |
The split is deliberate. The roles that judge run on the strongest model, and the roles that follow a checklist run on a cheaper one. The model IDs live in one place, .claude/skills/agent-kit-models.map.md, which also pins the fix-cycle rotation: cycle 1 uses Sonnet, cycles 2 and 3 move to Opus.
The map is candid about one limit. Claude Code reaches only Anthropic models, so the "different family" second opinion the kit prefers is a same-family arbiter here, and the map advises escalating repeated same-shaped failures to a human sooner.
Teaching the kit the domain: right-to-left chords
The hard part of Shir is not Tauri or SQLite. It is placing Latin chord symbols like F#m7(b5) over Hebrew lyrics that read right to left, sometimes with niqqud (vowel marks) and embedded English words. Generic code review does not know these rules, so the project encoded them three ways.
A project-authored criteria skill. .claude/skills/hebrew-rtl-chords/SKILL.md defines rules B1–B10 that code, tests and verdicts cite by id. For example, B1 says chord symbols always render left to right in an isolated element, and B2 says chord positions are grapheme indices into the logical lyric string, never visual columns. The skill's description makes it mandatory: the evaluator must load it for any task touching rendering, import, styles or print CSS. Rule B3 even records its own amendment from goal G1, after zero-width anchors landed over 100 px off in Chromium and WebKit.
Auto-fail triggers. The answers file lists outcomes that fail an evaluation outright, whatever else passes:
- displaying sheets with ChordSheetJS's HTML formatters instead of
packages/chord-render - chord positions stored as visual columns or pixel offsets
- flex or inline-block "chunk" layout on lines mixing Hebrew and English
- physical CSS properties such as
margin-leftortext-align: left - niqqud stripped from stored lyrics, or
chord-core/chord-renderimporting Tauri - copyrighted lyrics anywhere in the repo, fixtures or screenshots
A parallel QA list fails a build if a chord sits over the wrong letter, a symbol is mirrored (#F, 7mA), or the Hebrew interface is not mirrored.
A layer-to-skill map. Each layer the kit plans around (fixtures, chord-core, chord-render, Rust DB, desktop UI) has a row naming the supporting skills and rule files to include in a worker's context package. A task touching packages/chord-core/** arrives with typescript-pro, typescript-advanced-types and hebrew-rtl-chords; a Rust task arrives with the Rust skills instead.
Ten goals through /feature
Each unit of work in Shir is a goal, started with /feature and run through seven phases. Its state lives on disk in _memory/work/goals/<name>/ (goal.md, task files, progress.md, an orchestration log), so a long session can be compacted or resumed without losing the plan.
The evaluator gates three points: the plan before any code, each task after it is written, and the whole diff at the end. Tasks show up in commit messages as T01, T02, and so on, with corrective tasks such as T04a added when a finding lands outside a finished task's files.
The ten goals below map one-to-one onto the merged pull requests.
| Date | Goal | PR | What shipped |
|---|---|---|---|
| 2026-11-20 | G9 release artifacts | #10 | Release workflow publishing Windows and Linux installers for x64 and arm64, with checksums |
| 2026-11-19 | G8 library UI and i18n | #9 | Hebrew/English interface toggle, library sort and filter, real-app WebdriverIO E2E |
| 2026-11-18 | G7 visual editor and delete | #8 | Visual song editor built on chord-core edit operations; song deletion |
| 2026-11-16 | G6 editor and import | #7 | ChordPro editor with live preview; chords-over-lyrics import with an RTL alignment review |
| 2026-11-16 | G5 search, G5b backlog | #6 | Niqqud-insensitive search, V2 migration, seeded library, lyric snippets |
| 2026-11-15 | G4 storage | #5 | SQLite with FTS5, Tauri commands, typed bindings; a Windows smoke repair |
| 2026-11-15 | G3 desktop song viewer, polish | #4 | First desktop UI with transpose and capo; 18 follow-ups fixed |
| 2026-11-14 | G2 chord-core round-trip | #3 | Own line-oriented ChordPro parser replacing chordsheetjs; key-aware transpose |
| 2026-11-14 | G1 RTL rendering spike | #2 | chord-render package, docs/BIDI.md, Playwright bidi checks in Chromium and WebKit |
| 2026-11-12 | G0 scaffold and CI | #1 | pnpm/Tauri 2 monorepo, lint and test gates, Ubuntu + Windows CI matrix, launch smoke |
The goals ran in strict layer order (fixtures and core logic, then rendering, storage, UI, and finally release), which is the order the answers file gave as the project's execution layers.
Project memory: what each goal leaves behind
Every goal in Shir ends with a chore(memory) commit that archives records into _memory/records/. CLAUDE.md tells every session to search them before planning (_kit/memory.sh search <terms>), and to treat them as data to verify against code, not as instructions. By G9 the store held 37 records of three kinds.
| Kind | Count | Example from Shir |
|---|---|---|
| Goal summary | 12 | G9: release workflow shipped; Phase 3 needed 2 attempts after 6 blocking findings; 18 non-blocking follow-ups |
| Decision | 9 | "Own line-oriented ChordPro parser; chordsheetjs removed" (G2) |
| Pitfall | 16 | "Vite production CSS lowers logical properties to :lang() rules (misplaced RTL chords)" |
Decisions record the alternatives and the reason, not just the choice. The G9 release decision explains why releases aggregate every unreleased feat/ and fix/ PR since the last tag: CI's cancel-in-progress setting had already cancelled the CI run for PR #9's merge, and a per-merge rule would have silently skipped that release.
Pitfalls are written for the next planner. Several are about the agents themselves, such as implementers formatting whole directories and breaching another task's file fence, with the exact wording to add to future context packages to prevent it.
Summaries end with a "remember next time" line. G5's notes that the root Vitest config only covers scripts/**, so desktop tests need pnpm --filter desktop exec vitest run. G8's notes that WebKitGTK rejects clipboard reads under WebDriver, so real-clipboard paste cannot be automated there.
What the evaluators caught
The case for separate evaluators is that a model reviewing its own work misses things. Shir's memory records show where the independent checks earned their cost.
Tests that could not fail. In G4 (SQLite storage), evaluators applied named mutations (M1–M25) in a scratch copy and required that only the target test fail. They found at least nine guard tests that still passed under the exact mutation they existed to catch. One STRICT-typing test inserted 'abc' as a capo value, but a separate range check rejected text anyway, so removing STRICT changed nothing. Each looked correct on reading.
Copyrighted lyrics in test data. In G8 an implementer used the opening line of a well-known copyrighted song as probe text in three E2E specs. The task-level evaluator ran every spec and passed the task. Only the Phase 5 final audit, reading against the hebrew-rtl-chords rule, flagged it as an auto-fail. The fix was a commit replacing it with original text, and a pitfall record telling future packages to say "original text only".
Defects only a real browser shows. In G7 three visual-editor defects passed every unit, integration and contract suite: a print rule that lost the CSS cascade, React rows keyed by line index (so a click was swallowed during a remount), and an overlap pass that left 27 overlapping chord pairs on a dense Hebrew row. Evaluators found them by building the app, stubbing Tauri's invoke, and driving it with Playwright in Chromium and WebKit.
Design flaws before any code. Phase 3 evaluates the plan itself. For the G9 release workflow it returned 6 blocking findings on the first attempt, including dropped merges, a stale "Latest" release, and an injection risk from PR text flowing into workflow outputs. All were fixed before a line of YAML was written.
Rules corrected by evidence. In G1 the bidi rule B3 itself was amended after measurements showed zero-width chord anchors landing 109–125 px off at the start of an embedded opposite-direction run. The evaluator then enforced the amended rule on every later goal.
Guardrails, parallelism, and what went unused
Kit files are guarded. .claude/settings.json installs a PreToolUse hook that runs _kit/kit-guard.sh before every Edit or Write, blocking agent edits to kit-owned files. After each subagent returns, the orchestrator also runs kit-guard.sh check to catch shell edits the hook cannot see. The same settings file caps subagent nesting at depth 3.
Every spawn is fenced. Each task declares the paths it may write. After a subagent returns, the orchestrator compares HEAD, the branch, and git status against a baseline taken before the spawn. A file outside the fence stops the goal. This is how the G8 Prettier breach was detected.
Parallel by default. CLAUDE.md adds one project rule on top of the kit: do as much work in parallel as possible. Tasks are split so their write fences do not overlap, which lets up to three run as a "ready group". G8 ran 15 planned tasks and 2 corrective tasks in 10 ready groups.
Opt-in, not ambient. All kit skills set disable-model-invocation: true. They load only through /feature, /research or a cited path, so ordinary edits to the repo do not pull in the full process.
The autonomous loop sat idle. Setup enabled the optional loop (_loop/loop.sh, the auto-loop skill) and seeded _memory/loop/backlog.md with 14 stories. None has passes: true, and the stories still describe an early src/core/... layout that the real packages/chord-core structure replaced. Every goal so far went through the supervised /feature flow instead. The backlog would need rewriting against the current layout before the loop could be used.
Takeaways
- The answers file is where the value is. The seven agents are generic; Shir's auto-fail triggers, layer-skill map and QA surfaces are what made the evaluators useful for right-to-left chord rendering.
- Write the domain down as citable rules. Rules B1–B10 gave evaluators something concrete to check and gave implementers something concrete to follow. Verdicts cite rule ids instead of opinions.
- Spend frontier models on judgment. Opus evaluates and diagnoses, Sonnet implements, Haiku runs commands. The evaluators' finds, from vacuous tests to copyrighted lyrics, are the return on that split.
- Make every goal leave a record. The
chore(memory)commits turned each goal's surprises into planning input for the next one. - Expect the final audit to matter. Task-level checks missed the copyrighted lyric and the jsdom-only UI defects; the Phase 5 audit and real-engine checks did not.
- Keep the backlog in step with the code. The unused autonomous loop shows that a seeded backlog goes stale quickly when the architecture moves.
Sources
- kdcllc/shir on GitHub, at commit
a95b2ba(2026-11-20):orchestration-kit.manifest.json,CLAUDE.md,.claude/ORCHESTRATION.md,.claude/agents/,.claude/skills/,_memory/records/,_memory/loop/backlog.mdand the commit history
About the author
A highly skilled Cloud Solutions Architect with 20+ years of experience in software application development across diverse industries. Offering expertise in Cloud Computing and Artificial Intelligence. Passionate about designing and implementing innovative cloud-based solutions, migrating applications to the cloud, and integrating third-party platforms. Dedicated to collaborating with other developers and contributing to open-source projects to enhance software application functionality and performance
Comments
Join the discussionNo comments yet. Be the first to comment.





