feat: add manifest integration with multi-source deduplication and discovery modes#29
Open
mkellerman wants to merge 17 commits intomainfrom
Open
feat: add manifest integration with multi-source deduplication and discovery modes#29mkellerman wants to merge 17 commits intomainfrom
mkellerman wants to merge 17 commits intomainfrom
Conversation
- Add bmad-method as local file dependency - Add fs-extra for file system operations - Setup complete for manifest integration (Story 4)
- Add ManifestCache class wrapping BMAD-METHOD's ManifestGenerator - Support multi-source bmad_roots with priority-based deduplication - Implement manifest generation, loading, and validation - Add TypeScript declarations for bmad-method CommonJS module - Priority system: project (1) > user (2) > git remotes (3+) - Auto-detect modules, walk directories, merge resources - Hash-based file integrity checking support Story 5: ManifestCache Implementation - COMPLETE
- Add ManifestCache member to ResourceLoaderGit class - Update listAgentsWithMetadata() to use manifest-based loading with fallback - Update listWorkflowsWithMetadata() to use manifest-based loading with fallback - Add feature flag BMAD_USE_MANIFESTS (defaults to true) - Extract runtime scanning into private fallback methods - Maintain backward compatibility with graceful degradation Story 7: ResourceLoaderGit Integration - COMPLETE
Update package.json to use bmad-method@alpha from npm Fix ManifestGenerator import path resolution for npm package Use node:fs/promises for file operations Performance: 102ms agents with generation, 0.22ms workflows with cache
Generate per-source manifests to cache directory Write deduplicated results to merged directory Source directories never modified Performance: 100ms first load, sub-millisecond cached
- Add ambiguous result types (AmbiguousAgentResult, AmbiguousWorkflowResult) - Detect multiple matches when no module filter provided - Return formatted guidance for LLM with 95% confidence instruction - Include match metadata: module, agent, workflow, descriptions - Add composite keys (module:agent:workflow, module:agent) - Create comprehensive test suite (8 new unit tests) - Fix test path resolution (use process.cwd() instead of __dirname) - Auto-clean test results before each run to prevent stale data All tests passing (211/211 - 100%)
Add 4 discovery modes for controlling which BMAD sources are loaded: - strict: Only git remotes from CLI (no local/user) - local: Only project root (no user/git) - user: Only ~/.bmad (no project/git) - auto: All sources with priority (project > user > git) [default] Changes: - Add DiscoveryMode type to types/index.ts - Add --mode CLI argument to cli.ts - Add shouldIncludeSource() filtering in resource-loader.ts - Update manifest-cache.ts to respect discovery mode - Add 'local' test mode for test isolation in tests This enables controlled source discovery for development, testing, and production scenarios where source isolation is needed.
Extend manifest system to support tools and tasks alongside agents and workflows: - Add tool-manifest.csv and task-manifest.csv generation - Add Tool and Task types with module, path, standalone fields - Add getAllTools() and getAllTasks() to ManifestCache - Add listToolsWithMetadata() and listTasksWithMetadata() to ResourceLoader - Add virtual resource endpoints for bmad://_cfg/tool-manifest.csv and task-manifest.csv - Add generateToolManifest() and generateTaskManifest() to BMADEngine - Add manifest-generator.d.ts type declarations for local generator - Add test script for virtual manifest resources This completes the manifest system for all BMAD artifact types (agents, workflows, tools, tasks) with priority-based deduplication.
Add comprehensive test to verify ManifestCache never modifies source directories: - Verify no _cfg directory is created in source - Verify cache generates manifests (not source) - Test irregular bmad roots (module at root level) - Test cache regeneration from incomplete source manifests This ensures manifest generation always writes to cache directory (~/.bmad/cache/manifests) and never modifies source repositories, preventing git status pollution and maintaining source integrity.
Add comprehensive test infrastructure for manifest features: Test Scripts: - test-discovery-modes.mjs: Validates all 4 discovery modes (strict/local/user/auto) - test-workflow-extraction.mjs: Verifies workflow extraction from agent XML menu items - verify-workflow-deduplication.mjs: Tests workflow deduplication with module:name keys Unit Tests: - manifest-deduplication.test.ts: Tests tool/task priority-based deduplication Test Fixtures: - tests/fixtures/custom/: Complete test fixture with agents, tools, tasks, workflows - Includes pre-generated manifests for testing multi-source deduplication - Simulates user ~/.bmad source with lower priority items This validates that: - Priority-based deduplication works correctly (project > user > git) - Discovery modes properly filter sources - Workflows are extracted from agent menu items - All manifest types (agents, workflows, tools, tasks) deduplicate correctly
Development improvements for better visibility and debugging: - index.ts: Add version display from package.json on server startup These changes improve the development experience by showing: - Server version on startup for issue reporting - Cache directory structure and source mappings - Per-source and merged manifest locations - Sample manifest content for verification Helps developers understand the multi-source manifest system behavior.
Contributor
✅ PR Title Valid✨ MINOR version bump detected Current version: |
…al file The build script was failing because it tried to copy a non-existent src/core/manifest-generator.js file. Instead, we now dynamically import ManifestGenerator from the bmad-method npm package at runtime. Changes: - Remove manifest-generator.js copy from build script - Update manifest-cache.ts to import from bmad-method package - Remove manifest-generator.d.ts (no longer needed) - Use dynamic import with proper path resolution - Fix unbound-method linting error by using module reference This fixes the CI build failures while maintaining all functionality.
…ackages - Added copySourceManifestsToCache() method to handle pre-existing manifests - Updated ensureManifests() to check source directory before generating - Fixes test failures where fixtures have manifests but cache expects generation - Enables support for pre-built BMAD packages with bundled manifests - All 211 tests now passing (100% success rate)
Contributor
🔍 Preview Build ReadyA preview build has been created for this PR. Test this PRAdd this to your MCP configuration file: {
"mcpServers": {
"bmad": {
"command": "npx",
"args": ["-y", "git+https://github.com/mkellerman/bmad-mcp-server#pr-29"]
}
}
}
|
- Fix race condition causing Git sources to be resolved 3 times - Added Promise-based lock pattern (sourcesPromise) to ManifestCache - Ensures only one Git resolution happens even with parallel calls - Split getSources() into public method with lock and private loadSources() - Implement --mode argument parsing in MCP server - Parse --mode=<mode> in index.ts entry point - Pass discoveryMode to server and engine constructors - Add logging to show active discovery mode at startup - Add comprehensive directory exclusion list - Exclude .git, node_modules, cache, build, dist, and 30+ common dirs - Applied to manifest-cache.ts detectModules() and walkDirectory() - Applied to resource-loader.ts detectPathType() module scanning - Add test file for strict mode validation (test-strict-mode.mjs) This fixes issues where: 1. Git sources were resolved 3 times during parallel initialization 2. --mode=strict flag was ignored, defaulting to 'auto' mode 3. System/cache directories were scanned for BMAD modules
Added getSourceCount() and getModuleNames() methods to ManifestCache and ResourceLoaderGit to provide better visibility into what's loaded. Changes: - ManifestCache: Added getSourceCount() and getModuleNames() methods - ResourceLoaderGit: Added wrapper methods delegating to ManifestCache - server.ts: Updated start() to show 3-line logging format: * 'BMAD MCP Server started' * 'Loaded: X source(s), Y resources' * 'Loaded: Z module(s), A agents, B workflows' - Proper plural handling for sources and modules Provides clear observability of multi-source loading and modules.
- Add fallback to check for standalone workflows when no agent offers the workflow - Fixes workflow-init and workflow-status execution issues - Add e2e test to verify standalone workflow execution - Test confirms workflow-init can now be executed directly
- Created bmad-cli tool (src/cli-tester.ts) for command-line testing and debugging - Implemented JSON-RPC executor (src/utils/jsonrpc-executor.ts) with stdio communication - Added support for multiple --path options for multi-source configuration - Implemented 5 discovery modes: strict, local, user, first (default), auto - Fixed priority order: CLI args (1) > ENV (2) > local (3) > user (4) - Added BMAD_ROOT ENV variable support for git URLs - Eliminated Node.js deprecation warning with shell: false and custom argument parser - Enhanced server ready detection to match both 'module' and 'modules' - Updated manifest-cache.ts to implement first mode with proper priority filtering - Added comprehensive documentation (CLI Testing Guide, Test Strategy) CLI Features: - tools: List MCP tools - resources: List MCP resources - agents: List BMAD agents - workflows: List BMAD workflows - read-agent/read-workflow: Get details - exec-agent/exec-workflow: Execute with message - call: Generic tool execution - jsonrpc: Raw JSON-RPC requests Discovery Mode Behavior: - strict: Only CLI --path sources (git remotes) - local: Only project root - user: Only ~/.bmad - first: Use first source found (CLI > ENV > local > user), single-source - auto: Merge all sources with priority-based deduplication Priority System: - Priority 1 (highest): Git remotes from CLI --path args (explicit user choice) - Priority 2: ENV variables (BMAD_ROOT with git URL) - Priority 3+: Local project bmad (if exists) - Priority N+1 (lowest): User ~/.bmad Testing: - All 8 comprehensive tests passing - No deprecation warnings - Correct priority semantics validated - All discovery modes functional Breaking Changes: None Dependencies Added: commander@^12.1.0
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR implements the complete manifest integration system for bmad-mcp-server, enabling multi-source BMAD content discovery with priority-based deduplication and flexible discovery modes.
Features
1. Discovery Modes (Source Filtering)
Controlled via
--modeCLI argument for development, testing, and production scenarios.2. Tool and Task Manifest Support
bmad://_cfg/tool-manifest.csvandbmad://_cfg/task-manifest.csv3. Manifest Cache Source Protection
~/.bmad/cache/manifests/{hash}/_cfg/4. Ambiguous Response Handling
5. Comprehensive Test Infrastructure
Changes
Core Implementation (6 commits)
feat: implement ambiguous response handling (5d7d66c)
feat: add discovery mode for source filtering (b240ed2)
feat: add tool and task manifest support (bc721ed)
test: add source protection verification (b4d6919)
test: add deduplication and discovery mode tests (1acdd75)
chore: add development and debugging enhancements (b436ef2)
Bug Fixes (committed with ambiguous response feature)
Test Results
✅ All 211 tests passing (100% pass rate)
Breaking Changes
None - all changes are backwards compatible.
Migration Guide
No migration needed. Discovery mode defaults to
auto(current behavior).Checklist
Version Impact
This PR uses
feat:prefix, which will trigger a minor version bump (e.g., 3.0.0 → 3.1.0).✨ An alpha pre-release will be created automatically on merge for testing before stable release.