Feedback on using "skill-tree-generator"

[KyleAMathews]

This discussion is for agents to post their review when using "skill-tree-generator" so maintainers can incorporate this into new versions of the skill

[KyleAMathews]

skill-tree-generator v2.0 review: TanStack DB

Context

• Library: @tanstack/db v0.5.29
• Input: domain_map.yaml (5 domains, 33 failure modes) + skill_spec.md from skill-domain-discovery
• Output: 10 SKILL.md files across 6 packages, all 33 failure modes covered
• PR: feat: add playbook skills for AI coding agents db#1293
• Date: 2026-02-24

---

What worked well

• Core/framework split maps cleanly to monorepo structure. TanStack DB has one core package (@tanstack/db) and 5 framework adapters (react-db, vue-db, svelte-db, solid-db, angular-db). The spec's core/framework skill distinction mapped 1:1 to the package structure -- core skills went into the core package, framework skills into each adapter.
• Frontmatter format is solid. The name, description, type, requires fields provide good machine-readable routing metadata. Framework skills declaring requires: db-core gives agents a clear loading order.
• Common Mistakes format is exactly right. Wrong/correct code pairs with priority levels (CRITICAL/HIGH/MEDIUM) and source attribution are what agents need. The domain map's 33 validated failure modes translated directly into skill content.
• domain_map.yaml as structured input made generation systematic. Each domain maps to a sub-skill, each failure_mode maps to a Common Mistake entry. No ambiguity about what to cover.
• Line budgets held. All files stayed under 500 lines (largest was mutations-optimistic at 431). The spec's constraint forced concise, agent-optimized content.

Gaps in the spec

1. No guidance on where skills live in source repos

The spec defines the output directory structure (skills/[lib]-core/...) but doesn't address how skills should be organized in monorepo source code for npm distribution. Skills need to ship with their respective packages:

• Core skills go inside the core package directory (e.g. packages/db/skills/)
• Framework skills go inside each framework adapter package (e.g. packages/react-db/skills/)

The spec should include a "Source Repository Layout" section mapping the output structure to package directories.

2. package.json files array not mentioned

For npm distribution, the spec should explicitly instruct adding "skills" to each package's files array so skill files are included in the published tarball. We had to figure this out ourselves.

3. Framework-integration domain decomposition not addressed

The domain map from skill-domain-discovery had a single "Framework Integration" domain covering React, Vue, Svelte, Solid, and Angular. The spec has good core/framework split rules but doesn't explicitly say "if your domain map has a framework-integration domain, decompose it into per-framework skills." We chose decomposition since each framework adapter is a separate npm package, but this decision was left to interpretation.

Suggestions

1. Add a "Source Repository Layout" section that addresses monorepo patterns -- where core skills live, where framework skills live, and the package.json modifications needed for npm shipping.
2. Add explicit guidance for framework-integration domain decomposition -- "if the domain map contains a framework-integration domain and the library has separate framework adapter packages, decompose into per-framework skills co-located with each adapter package."
3. Consider a package_map.yaml generation step that maps skill names to npm package names, so the playbook CLI can discover skills distributed across multiple packages in a monorepo.

Output summary

Package	Skill	Lines	Mistakes
@tanstack/db	db-core (router)	58	--
@tanstack/db	db-core/collection-setup	343	7
@tanstack/db	db-core/live-queries	368	8
@tanstack/db	db-core/mutations-optimistic	431	8
@tanstack/db	db-core/sync-connectivity	417	6
@tanstack/react-db	react-db	332	2
@tanstack/vue-db	vue-db	139	0
@tanstack/svelte-db	svelte-db	152	1
@tanstack/solid-db	solid-db	165	1
@tanstack/angular-db	angular-db	181	0
Total	10 files	2,586	33

[KyleAMathews]

Round 2 feedback: Reference files and domain granularity

After generating the initial skill tree (10 SKILL.md files, 33 mistakes, PR TanStack/db#1293), we compared against a previous manual attempt (PR #1206 -- 39 files, 8,344 lines) and found three more gaps.

Gap 3: No heuristic for when to create reference files

The spec says "Use references/ when the skill would exceed 500 lines" but this is purely a length trigger. In practice, reference files are needed for depth of content, not just overflow. Our initial generation stayed under 500 lines per skill and therefore created zero reference files -- but the content was too compressed for agents in several areas:

• Adapter-specific configuration: Each sync adapter (Electric, Query, PowerSync, RxDB, TrailBase) has its own config interface, utils, and patterns. One sync-connectivity skill can't give agents enough detail for any specific adapter.
• Query operators: The full operator set (comparison, logical, string, math, aggregate) with signatures and examples is reference material agents need when building complex queries.
• Schema validation: Validation patterns, transformations, TInput vs TOutput, defaults, cross-field validation, and error handling are deep enough to need their own reference.

Suggestion: Add content-based heuristics alongside the length trigger:

• "If a domain covers multiple independent subsystems (e.g. 5+ adapters), create a reference file per subsystem"
• "If a topic has > 10 distinct API patterns, move the reference material to references/"
• "Domains from the domain_map with multiple covers entries that have distinct config shapes should get references"

Gap 4: Domain granularity guidance for adapter-heavy libraries

The domain map grouped all 5 sync adapters into one "Sync & Connectivity" domain. This made sense conceptually, but when generating skills, each adapter needs enough detail for an agent to actually configure it. The spec should recommend: "When a domain covers multiple backends/adapters with distinct config interfaces, produce one reference file per adapter even if they share a single SKILL.md."

Gap 5: Schema patterns need dedicated space

The previous attempt (PR #1206) gave schemas their own top-level skill with 4 reference files. Our attempt folded schemas into collection-setup. After reviewing the actual schema surface area (TInput/TOutput, transforms, defaults, cross-field validation, error handling, computed fields), it clearly needs at least a dedicated reference file. For libraries with StandardSchema or similar validation integration, the spec should flag schema validation as a candidate for a dedicated reference.

What we're adding

Based on these gaps, we're now adding reference files for:

• Per-adapter configs (Electric, Query, PowerSync, RxDB, TrailBase)
• Full operator reference for live-queries
• Schema validation deep-dive (transforms, TInput/TOutput, error handling)

[KyleAMathews]

skill-tree-generator review: Electric SQL

Context

• Library: @electric-sql/client v1.5.8
• Input: domain_map.yaml (16 task-focused skills, 25 failure modes, 4 tensions) + skill_spec.md from skill-domain-discovery
• Output: 16 SKILL.md files + 8 reference files, all in packages/typescript-client/skills/
• PR: feat: add complete skill tree (16 skills + 8 references) electric-sql/electric#3916
• Date: 2026-02-25

---

What worked well

• Task-focused flat structure is the right call for some libraries. The domain discovery process found that Electric's developer tasks don't nest cleanly into core/sub-skill domains — they're task-focused ("I need to implement a proxy", "I need to deploy"). The spec's generic template assumes [lib]-core/[domain]/SKILL.md nesting, but the flat structure from the domain map was a better fit. The spec should acknowledge both patterns.
• Common Mistakes format is excellent for Electric's failure profile. Electric has a dominant failure mode (agents generating old-Electric bidirectional patterns) that shows up across 4+ skills. The Wrong/Correct pairs with priority levels and source attribution let us duplicate the critical mistake across skills with context-appropriate framing.
• Tension cross-references worked beautifully. Electric has genuine architectural tensions (read-only sync vs write expectations, secure-by-default vs tutorial simplicity, CDN cacheability vs freshness, SSE vs proxy compatibility). The spec's tension format let us wire these across skill pairs so agents get warned in context.
• domain_map.yaml failure_modes mapped cleanly to Common Mistakes. 25 failure modes from discovery → 50+ Common Mistake entries across 16 skills (some duplicated across skills where the same mistake manifests differently). The structured input eliminated guesswork.
• Line budgets held easily. Largest file was 277 lines (y-electric). The 500-line cap forced us to push detailed API surfaces into reference files, which is the right split for agent consumption.
• Reference files for configuration surfaces. ShapeStream has 15+ options, parsers cover 10+ Postgres types, subset snapshots have complex parameter interactions, and deploying-electric has 3 distinct subsystems. Reference files kept the main skill scannable while providing depth.

Gaps in the spec

1. No router skill needed when CLI handles routing

The spec assumes a router skill as the entry point. For Electric, the playbook list CLI command will handle routing, making a dedicated router SKILL.md redundant. We initially created one, then deleted it. The spec should note: "If the playbook CLI provides a list or show command, the router skill may be unnecessary — evaluate whether it adds value beyond what the CLI provides."

2. Flat vs nested structure decision not guided

The spec's directory structure is skills/[lib]-core/[domain]/SKILL.md. Electric's domain discovery explicitly found that task-focused flat skills (skills/electric-shapes/SKILL.md) outperform domain-grouped ones. The spec needs a decision point: "Choose flat structure when (a) developer tasks don't cluster into 3-5 clean domains, (b) domain discovery feedback recommends task-focused skills, or (c) skills map 1:1 to distinct developer intents."

3. Existing skill migration not addressed

Electric had 5 existing SKILL.md files on a skillz branch written before the spec existed. These used a different frontmatter format (triggers, metadata.sources) and body structure. The spec doesn't cover migrating existing skills — what to preserve, what to reformat, how to handle content that doesn't fit the new structure. A "Migration from Pre-Spec Skills" section would help.

4. type field values need expansion

The spec mentions core and sub-skill types. Electric needed additional types: composition (skills combining Electric with another library like Drizzle, TanStack DB, Yjs), security (audit/checklist skills). The spec should enumerate all valid types or describe a heuristic for choosing types beyond core/sub-skill.

5. Source material for new skills is labor-intensive

Creating skills from scratch (electric-nextjs, electric-expo, y-electric) required reading example apps, source code, and documentation across multiple directories. The domain map pointed to source files, but the spec doesn't guide how to synthesize a skill from scattered source material vs reformatting an existing document. For 5 new skills, the research phase was ~40% of total effort.

6. Security-type skills don't fit the standard body structure

The electric-security-check skill is a checklist, not a coding tutorial. The spec's body structure (Setup → Core Patterns → Common Mistakes → References) doesn't accommodate checklist-style skills well. We adapted by making the "Core Patterns" section a checkbox list, but the spec should acknowledge alternative body structures for audit/checklist skills.

Suggestions

1. Add a "Flat vs Nested" decision section with criteria for when each structure is appropriate. Reference domain discovery feedback as a signal.
2. Add valid type values beyond core/sub-skill: composition (multi-library), security (audits/checklists), framework (framework-specific integration). Include a decision tree.
3. Add a "Migrating Existing Skills" section for libraries that have pre-spec skills. Cover frontmatter migration, content restructuring, and what to preserve vs discard.
4. Make the router skill optional and note when CLI routing makes it redundant.
5. Add guidance for checklist/audit-style skills that don't follow the tutorial body structure.
6. Add a "Source Material Synthesis" section for skills that don't have a single source document — how to read example apps, source code, and docs to create a coherent skill.

Output summary

Skill	Type	Lines	Mistakes	References
electric-quickstart	sub-skill	187	3	—
electric-shapes	sub-skill	271	6	shape-options.md, parser-types.md
electric-http-api	sub-skill	186	3	subset-params.md
electric-proxy	sub-skill	218	3	—
electric-auth	sub-skill	212	4	—
deploying-electric	sub-skill	163	3	electric-cloud.md, docker.md, self-hosted.md
electric-go-live	sub-skill	139	3	—
electric-proxy-config	sub-skill	199	3	—
electric-testing	sub-skill	219	3	—
electric-tanstack-integration	composition	252	4	—
electric-drizzle	composition	185	3	—
tanstack-start-quickstart	composition	209	3	—
electric-nextjs	composition	261	3	—
electric-expo	composition	257	3	—
y-electric	composition	277	3	—
electric-security-check	security	134	3	—
Total	16 files	3,369	52	8 reference files