Skip to content

Conversation

@cxxxr
Copy link
Member

@cxxxr cxxxr commented Dec 22, 2025

Description

Add tree-sitter based call graph providers for Python, JavaScript/TypeScript, and Go languages. This enables Living Canvas to visualize function call graphs for these languages in addition to Common Lisp.

New Features

  • Provider Registry System: Language-agnostic provider management with priority-based selection
  • Language Detection: Automatic language identification by major mode or file extension
  • Python Provider: Function/method extraction and call relationship analysis via tree-sitter
  • JavaScript/TypeScript Provider: Support for function declarations, arrow functions, and method definitions
  • Go Provider: Support for functions and methods
  • New Commands:
    • living-canvas-current-buffer - Multi-language support with auto-detection
    • living-canvas-diagnose - Troubleshoot provider status
    • living-canvas-reload-providers - Manual provider registration

Supported Languages

Language File Extensions Provider
Common Lisp .lisp, .cl, .asd micros (existing)
Python .py, .pyw tree-sitter
JavaScript .js, .mjs, .cjs, .jsx tree-sitter
TypeScript .ts, .tsx, .mts, .cts tree-sitter
Go .go tree-sitter

Future Work (separate PR)

  • Project-wide analysis for tree-sitter languages (currently single-file only)

AI Usage Disclosure

Did you use LLMs/AI tools for this PR?

  • No, this is 100% human-authored.
  • Yes, AI-assisted (Human-led logic, AI-assisted implementation).
  • Yes, AI-generated (Logic primarily derived from prompt/vibe).

Tooling Used: Claude Code (Claude Opus 4.5)


The "Human-in-the-Loop" Verification

1. Logic Walkthrough

The implementation follows a provider-based architecture for extensibility:

  1. Provider Registry (call-graph/registry.lisp): A central registry that manages multiple language providers. Each provider has a priority, and the registry selects the highest-priority provider that supports a given language/source.

  2. Language Detection (living-canvas/language-detection.lisp): Uses a two-tier detection strategy:

    • Primary: Check buffer's major mode (most reliable)
    • Fallback: Check file extension
  3. Tree-sitter Providers (Python, JS, Go): Each provider:

    • Compiles tree-sitter queries for function definitions and calls
    • Extracts function/method nodes with source locations and argument lists
    • Builds call edges by finding enclosing functions for each call site
    • Registers itself on load and via *after-init-hook* (handles Nix build timing)
  4. Command Flow: living-canvas-current-bufferdetect-languagefind-providerprovider-analyze → render graph

Key design decisions:

  • Providers register themselves to avoid central coordination
  • *after-init-hook* solves Nix build issue where grammars aren't available at compile time
  • Each provider is independent and can be loaded separately

2. Reviewer's Guide

High-risk areas:

  • find-enclosing-function in each provider - uses return-from to correctly exit the function (not just the loop)
  • Tree-sitter query syntax - queries must match the grammar's node types exactly
  • Nix flake LD_LIBRARY_PATH - all grammar paths must be included for runtime loading

Suggested focus for reviewer:

  • Verify tree-sitter queries work with real-world code (tested with sample files)
  • Check that provider registration timing works in different startup scenarios
  • Review CSS styles in buffer.lisp for new node types (class, method)

Related Issues

N/A (New feature based on spec in specs/001-multi-language-canvas/)

cxxxr and others added 3 commits December 23, 2025 00:37
Define feature specification for extending Living Canvas to support
multiple programming languages beyond Common Lisp. Key decisions:

- Pluggable language provider architecture
- Static analysis (AST/tree-sitter) as primary approach, LSP optional
- Initial support for Common Lisp, Python, and JavaScript/TypeScript
- Unified command interface across all languages

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…nguage support

Complete speckit workflow generating design documents for extending Living Canvas
to support Python and JavaScript/TypeScript via tree-sitter-based providers:

- plan.md: Technical context, constitution check, project structure
- research.md: Analysis approach decisions (tree-sitter over Python AST)
- data-model.md: Entity definitions for provider architecture
- quickstart.md: Step-by-step implementation guide (~760 LOC scope)
- tasks.md: 98 tasks organized by user story (59 in MVP scope)
- contracts/provider-interface.lisp: Formal provider API contract
- contracts/tree-sitter-queries.md: Query patterns for Python/JS/TS

Also applies spec.md remediation edits from analysis phase:
- Align Python analysis method with plan (tree-sitter)
- Scope module/system-level scenarios to Phase 2

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…t, TypeScript, and Go

Add tree-sitter based call graph providers for Python, JavaScript/TypeScript,
and Go languages. This enables Living Canvas to visualize function call graphs
for these languages in addition to Common Lisp.

Changes:
- Add provider registry system in call-graph extension for managing multiple
  language providers with priority-based selection
- Add language detection module that identifies file language by major mode
  or file extension
- Implement tree-sitter Python provider with function/method extraction and
  call relationship analysis
- Implement tree-sitter JavaScript/TypeScript provider supporting function
  declarations, arrow functions, and method definitions
- Implement tree-sitter Go provider for functions and methods
- Add living-canvas-current-buffer command for multi-language support
- Add living-canvas-diagnose command for troubleshooting provider status
- Add living-canvas-reload-providers command for manual provider registration
- Update flake.nix with tree-sitter grammars for Python, JS, TS, and Go
- Add comprehensive test suites for all new components
- Add sample files for testing each supported language

The new living-canvas-current-buffer command automatically detects the
programming language and selects the appropriate provider. Common Lisp
continues to use the existing micros-based provider.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@code-contractor-app
Copy link
Contributor

code-contractor-app bot commented Dec 22, 2025

❌ Code Contractor Validation: FAILED

✓ Code Contractor Validation Result
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

📊 Statistics:
  Files Changed:    33
  Lines Added:      6492
  Lines Deleted:    17
  Total Changed:    6509
  Delete Ratio:     0.00 (0%)

Status: FAILED ❌

🤖 AI Providers:
  - codex — model: (Codex default)

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
📋 Validation Rules:

❌ max_files_changed
❌ max_total_changed_lines
✅ max_delete_ratio
✅ defpackage_rule
✅ file_structure_rule
✅ loop_keywords_rule
✅ naming_conventions_rule
✅ docstring_rule
✅ internal_symbol_rule
✅ error_handling_rule
✅ frontend_interface_rule
❌ functional_style_rule
✅ dynamic_symbol_call_rule
✅ alexandria_usage_rule
✅ macro_style_rule

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
⚠️ Violations Found (3):

[ERROR] max_files_changed
  Too many files changed: 33 > 10
  Limit: 10
  Actual: 33

[ERROR] max_total_changed_lines
  Too many lines changed: 6509 > 400
  Limit: 400
  Actual: 6509

[ERROR] functional_style_rule
  AI check failed: "functional_style_rule"
  ❌ Reason:
    Uses the dynamic global `*provider-registry*` as shared state instead of
    passing the registry explicitly via function arguments, violating the
    preference for explicit arguments over dynamic variables (and relying on
    `defvar`-based cross-function state).

Fix Suggestions

  • functional_style_rule

  • Fix for violation errors checked above

📚 About Code Contractor

Declarative Code Standards That Learn and Improve

Define domain-specific validation rules in YAML.
Your contracts document team knowledge and evolve into more accurate AI enforcement.

Want this for your repo?
Install Code Contractor

cxxxr and others added 2 commits December 23, 2025 02:50
Move tree-sitter call-graph providers from living-canvas to their
language-specific mode directories:
- python-provider.lisp → extensions/python-mode/call-graph-provider.lisp
- js-provider.lisp → extensions/js-mode/call-graph-provider.lisp
- go-provider.lisp → extensions/go-mode/call-graph-provider.lisp

Each mode now defines a subsystem (e.g., lem-python-mode/call-graph)
that can be loaded independently. Living-canvas depends on these
subsystems rather than bundling the providers directly.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- file_structure_rule: Move defvar before defclass in registry.lisp,
  initialize to nil and set after class definition
- docstring_rule: Add defgeneric declarations with docstrings for
  exported condition readers (provider-error-provider, etc.)
- functional_style_rule: Remove :register option from macro that used
  *provider-registry* directly; document global as well-documented exception
- macro_style_rule: Extract option parsing to %parse-provider-options
  helper function, keeping macro minimal

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Copy link
Contributor

@code-contractor-app code-contractor-app bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Contractor validation failed ❌ — see the sticky comment for full results.

(let ((provider (make-go-provider)))
(when (provider-ts-language provider)
;; Only register if not already registered
(unless (find-provider *provider-registry* :go)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Contractor: functional_style_rule

AI check failed: "functional_style_rule"

Reason:
Uses the dynamic global *provider-registry* as shared state instead of passing the registry explicitly via function arguments, violating the preference for explicit arguments over dynamic variables (and relying on defvar-based cross-function state).

@cxxxr cxxxr merged commit 86914d3 into main Dec 23, 2025
11 checks passed
vindarel added a commit to lem-project/lem-project.github.io that referenced this pull request Dec 23, 2025
mahmoodsh36 pushed a commit to mahmoodsh36/lem that referenced this pull request Dec 24, 2025
…t, TypeScript, and Go (lem-project#2058)

* feat(living-canvas): add multi-language support specification

Define feature specification for extending Living Canvas to support
multiple programming languages beyond Common Lisp. Key decisions:

- Pluggable language provider architecture
- Static analysis (AST/tree-sitter) as primary approach, LSP optional
- Initial support for Common Lisp, Python, and JavaScript/TypeScript
- Unified command interface across all languages

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(living-canvas): add implementation design artifacts for multi-language support

Complete speckit workflow generating design documents for extending Living Canvas
to support Python and JavaScript/TypeScript via tree-sitter-based providers:

- plan.md: Technical context, constitution check, project structure
- research.md: Analysis approach decisions (tree-sitter over Python AST)
- data-model.md: Entity definitions for provider architecture
- quickstart.md: Step-by-step implementation guide (~760 LOC scope)
- tasks.md: 98 tasks organized by user story (59 in MVP scope)
- contracts/provider-interface.lisp: Formal provider API contract
- contracts/tree-sitter-queries.md: Query patterns for Python/JS/TS

Also applies spec.md remediation edits from analysis phase:
- Align Python analysis method with plan (tree-sitter)
- Scope module/system-level scenarios to Phase 2

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(living-canvas): add multi-language support for Python, JavaScript, TypeScript, and Go

Add tree-sitter based call graph providers for Python, JavaScript/TypeScript,
and Go languages. This enables Living Canvas to visualize function call graphs
for these languages in addition to Common Lisp.

Changes:
- Add provider registry system in call-graph extension for managing multiple
  language providers with priority-based selection
- Add language detection module that identifies file language by major mode
  or file extension
- Implement tree-sitter Python provider with function/method extraction and
  call relationship analysis
- Implement tree-sitter JavaScript/TypeScript provider supporting function
  declarations, arrow functions, and method definitions
- Implement tree-sitter Go provider for functions and methods
- Add living-canvas-current-buffer command for multi-language support
- Add living-canvas-diagnose command for troubleshooting provider status
- Add living-canvas-reload-providers command for manual provider registration
- Update flake.nix with tree-sitter grammars for Python, JS, TS, and Go
- Add comprehensive test suites for all new components
- Add sample files for testing each supported language

The new living-canvas-current-buffer command automatically detects the
programming language and selects the appropriate provider. Common Lisp
continues to use the existing micros-based provider.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor(call-graph): move language providers to respective mode modules

Move tree-sitter call-graph providers from living-canvas to their
language-specific mode directories:
- python-provider.lisp → extensions/python-mode/call-graph-provider.lisp
- js-provider.lisp → extensions/js-mode/call-graph-provider.lisp
- go-provider.lisp → extensions/go-mode/call-graph-provider.lisp

Each mode now defines a subsystem (e.g., lem-python-mode/call-graph)
that can be loaded independently. Living-canvas depends on these
subsystems rather than bundling the providers directly.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(call-graph): address code-contractor review violations

- file_structure_rule: Move defvar before defclass in registry.lisp,
  initialize to nil and set after class definition
- docstring_rule: Add defgeneric declarations with docstrings for
  exported condition readers (provider-error-provider, etc.)
- functional_style_rule: Remove :register option from macro that used
  *provider-registry* directly; document global as well-documented exception
- macro_style_rule: Extract option parsing to %parse-provider-options
  helper function, keeping macro minimal

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants