Refactor: Architecture cleanup (Phases 1-2)

.claude/CLAUDE.md

@@ -87,15 +87,27 @@ Then navigate to `http://127.0.0.1:5004/` in the browser.
 2. `tool_call_processor.py`, `ai_model.py`, and `functions_definitions.py` define the function-call surface exposed to GPT models.
 3. `webdriver_manager.py` captures canvas screenshots for the vision workflow.
 4. `workspace_manager.py` and `log_manager.py` handle persistence and auditing.
-5. `style.css` and other assets shared with the frontend live here for Flask to serve.
+5. `config.py` centralizes server-side constants (workspace dirs, schema version, snapshot paths).
+6. `env_config.py` provides shared environment variable loading, replacing duplicated `load_dotenv` patterns.
+7. `route_helpers.py` contains extracted route helper functions for provider management and tool lifecycle.
+8. `style.css` and other assets shared with the frontend live here for Flask to serve.
 
 ## Client Highlights (`static/client/`)
 1. `main.py` bootstraps Brython and registers managers.
 2. `canvas.py`, `canvas_event_handler.py`, and `managers/` orchestrate SVG drawing, selection, undo, and edit policies.
 3. `drawables/` contains shape classes (point, segment, vector, triangle, rectangle, circle, ellipse, angle, etc.).
-4. `ai_interface.py`, `process_function_calls.py`, and `result_processor.py` coordinate chat responses and tool execution.
-5. `expression_evaluator.py`, `expression_validator.py`, and `result_validator.py` provide math parsing, validation, and error messaging.
-6. `client_tests/` plus `test_runner.py` implement the Brython test harness (register new tests in `client_tests/tests.py`).
+4. `ai_interface.py` orchestrates AI communication and request building, delegating to five extracted managers.
+5. `chat_ui_manager.py` handles message rendering, streaming display, and markdown parsing.
+6. `message_menu_manager.py` manages context menus, clipboard, and raw text storage.
+7. `tool_call_log_manager.py` provides tool call visualization and state tracking.
+8. `image_attachment_manager.py` manages the file picker, preview, and image modal.
+9. `tts_ui_manager.py` controls text-to-speech UI (read aloud, voice settings).
+10. `process_function_calls.py` and `result_processor.py` coordinate tool execution and result handling.
+11. `expression_evaluator.py`, `expression_validator.py`, and `result_validator.py` provide math parsing, validation, and error messaging.
+12. `managers/base_drawable_manager.py` provides a shared base for all drawable managers.
+13. `managers/visibility_manager.py` handles viewport culling extracted from Canvas.
+14. `rendering/base_telemetry.py` provides a shared telemetry base for renderers.
+15. `client_tests/` plus `test_runner.py` implement the Brython test harness (register new tests in `client_tests/tests.py`).
 
 ---
 

documentation/Project Architecture.txt

@@ -70,7 +70,7 @@ MatHud is an interactive mathematical visualization tool that combines a drawing
 8. DOM surfaces are layered as WebGL (highest `z-index` 20), Canvas2D (`z-index` 10), and SVG (base layer) inside `#math-container`, with inactive surfaces kept `pointer-events: none`. Labels remain on SVG to guarantee text clarity across backends.
 9. Runtime selection relies on the preference chain inside `create_renderer`: it first tries Canvas2D, falls back to SVG if Canvas2D fails, and finally attempts WebGL. Constructor errors are caught so the factory can continue down the chain.
 10. Feature flags remain available for diagnostics: `window.MatHudSvgOffscreen` (or `localStorage["mathud.svg.offscreen"]`) toggles SVG offscreen staging, and `window.MatHudCanvas2DOffscreen` (or `localStorage["mathud.canvas2d.offscreen"]`) enables Canvas2D layer compositing.
-11. Renderers expose `begin_frame`, `end_frame`, `peek_telemetry`, and `drain_telemetry` hooks so automated tests and performance harnesses can capture plan build/apply timings, skip counts, adapter events, and maximum batch depth.
+11. Renderers expose `begin_frame`, `end_frame`, `peek_telemetry`, and `drain_telemetry` hooks so automated tests and performance harnesses can capture plan build/apply timings, skip counts, adapter events, and maximum batch depth. Shared telemetry infrastructure lives in `base_telemetry.py`.
 
 ## Multi-Step Calculations
 - Expressions: evaluate mathematical expressions and functions
@@ -124,7 +124,7 @@ When the AI executes tool calls during a conversation turn, a collapsible "Tool
 - **Entry contents**: Each entry shows a status icon (checkmark for success, cross for error), the function name, a compact argument summary, and an error message when applicable.
 - **Final summary**: After the turn completes, the summary updates to "Used N tool(s)" with an optional "(M failed)" suffix if any calls returned errors.
 - **State lifecycle**: Tool call log state (`_tool_call_log_entries`, `_tool_call_log_element`, etc.) is reset at the start of each new user message and cleaned up in `_finalize_stream_message`.
-- **Implementation**: `AIInterface._add_tool_call_entries` records entries and updates the DOM; `_finalize_tool_call_log` writes the final summary. CSS classes are prefixed with `tool-call-log-*` (defined in `static/style.css`).
+- **Implementation**: `ToolCallLogManager` (extracted from `AIInterface`) records entries and updates the DOM; its `finalize` method writes the final summary. CSS classes are prefixed with `tool-call-log-*` (defined in `static/style.css`).
 - **Tests**: `static/client/client_tests/test_tool_call_log.py` covers argument formatting, entry element creation, dropdown lifecycle, accumulation across rounds, finalization, and state management.
 
 ---
@@ -167,6 +167,9 @@ static/
 ├── functions_definitions.py           # 70 AI function/tool definitions
 ├── ai_model.py                        # AI model configuration utilities
 ├── tool_call_processor.py             # Tool call processing utilities
+├── config.py                          # Centralized server-side constants
+├── env_config.py                      # Shared environment loading utility
+├── route_helpers.py                   # Extracted route helper functions
 ├── style.css                          # Frontend CSS styling
 └── client/                           # Custom Python modules for Brython (moved from Brython-3.11.3/Lib/site-packages/)
     # Note: Brython core (3.12.5), Math.js (14.5.2), and Nerdamer (1.1.13) now loaded from CDN
@@ -179,7 +182,12 @@ templates/
 
 static/client/
 ├── main.py                            # Brython initialization and entry point
-├── ai_interface.py                    # AI communication and UI management
+├── ai_interface.py                    # AI communication orchestration and request building (refactored, ~1,078 lines)
+├── chat_ui_manager.py                 # Message rendering, streaming display, markdown (extracted from ai_interface)
+├── message_menu_manager.py            # Context menus and clipboard (extracted from ai_interface)
+├── tool_call_log_manager.py           # Tool call visualization (extracted from ai_interface)
+├── image_attachment_manager.py        # Image attachment handling (extracted from ai_interface)
+├── tts_ui_manager.py                  # Text-to-speech UI (extracted from ai_interface)
 ├── markdown_parser.py                 # Comprehensive markdown parser with LaTeX support
 ├── canvas_event_handler.py            # Mouse/keyboard event handling with extensive error handling
 ├── canvas.py                          # Core SVG canvas management
@@ -245,6 +253,7 @@ static/client/
 │   ├── segments_area_renderable.py    # Area between one or two segments
 │   ├── shared_drawable_renderers.py   # Drawing helpers shared across all backends
 │   ├── style_manager.py               # Renderer style dictionaries
+│   ├── base_telemetry.py              # Shared telemetry base class for renderers
 │   ├── cached_render_plan.py          # OptimizedPrimitivePlan type for caching
 │   ├── canvas2d_primitive_adapter.py  # Canvas 2D primitive adapter
 │   ├── svg_primitive_adapter.py       # SVG primitive adapter with DOM pooling
@@ -257,8 +266,10 @@ static/client/
 │       ├── screen_offset_label_layout.py   # Label overlap resolution
 │       └── label_overlap_resolver.py  # Greedy overlap resolution algorithm
 ├── managers/                          # Specialized object managers
+│   ├── base_drawable_manager.py       # Base class for drawable managers
 │   ├── drawable_manager.py            # Central drawable orchestrator
 │   ├── drawable_manager_proxy.py      # Initialization dependency management
+│   ├── visibility_manager.py          # Drawable visibility control (extracted from Canvas)
 │   ├── drawables_container.py         # Drawable storage and organization
 │   ├── drawable_dependency_manager.py # Object dependency tracking
 │   ├── undo_redo_manager.py           # Undo/redo functionality
@@ -363,7 +374,7 @@ server_tests/
 ## Architecture Notes
 - **Brython Integration**: Extensive use of Python in the browser via Brython 3.12.5 framework (CDN)
 - **Facade Pattern**: `ProcessFunctionCalls` acts as facade for `ResultProcessor`, `ExpressionEvaluator`, `ResultValidator`
-- **Manager Pattern**: Specialized managers for each geometric shape type with centralized coordination
+- **Manager Pattern**: Specialized managers for each geometric shape type with centralized coordination; drawable managers inherit from `BaseDrawableManager`
 - **Error Handling**: Comprehensive try-catch blocks throughout client-side code
 - **Testing**: Dual testing system with server-side pytest and client-side Brython unittest
 - **Renderer Logic Tests**: `server_tests/client_renderer/` houses pytest suites that exercise plan caching, factory fallbacks, helper metadata, and color normalization using lightweight browser stubs. The harness runs through `run_server_tests.py`, which now injects `static/client` into `PYTHONPATH` so the renderer modules and shared utilities import correctly during server-side runs.
@@ -385,26 +396,28 @@ server_tests/
 - `static/client/canvas_event_handler.py` manages user interactions with the canvas.
 
 ## AI Interface Layer (`static/client/ai_interface.py`)
-- Core class `AIInterface` manages Brython-side communication between UI and backend.
+- Core class `AIInterface` manages Brython-side AI communication orchestration and request building (~1,078 lines, refactored down from ~2,339).
+- Five UI-focused concerns were extracted into dedicated managers:
+  - `ChatUIManager` (`chat_ui_manager.py`): message rendering, streaming display, and markdown formatting.
+  - `MessageMenuManager` (`message_menu_manager.py`): context menus and clipboard operations.
+  - `ToolCallLogManager` (`tool_call_log_manager.py`): tool call visualization and log dropdown.
+  - `ImageAttachmentManager` (`image_attachment_manager.py`): image attachment handling.
+  - `TTSUIManager` (`tts_ui_manager.py`): text-to-speech UI controls.
 - Instantiated in `main.py` and receives events from `CanvasEventHandler`.
 - Initializes client-side `WorkspaceManager` (from `static/client/workspace_manager.py`) and `FunctionRegistry`.
-- **Markdown Integration**: Initializes `MarkdownParser` for comprehensive text formatting and LaTeX rendering.
 - Maintains dictionary of available functions for math operations and canvas manipulation (via `FunctionRegistry`).
 - Handles function call results and maintains state between interactions.
-- **UI Element References**: References `send-button` for button control, `chat-input` for input field, `vision-toggle` for vision checkbox, and `ai-model-selector` for model selection.
 - Key methods:
   - `interact_with_ai`: Entry point for user interactions from UI.
   - `_send_prompt_to_ai`: Prepares JSON prompt (with canvas state, user message, tool results, vision settings from `document["vision-toggle"]`, AI model from `document["ai-model-selector"]`) and sends AJAX POST request to `/send_message` Flask endpoint.
   - `_process_ai_response`: Processes AI responses from backend. If tool calls are present, uses `ProcessFunctionCalls.get_results` and then sends results back to AI via `_send_prompt_to_ai`.
-  - `_parse_markdown_to_html`: Converts markdown text to HTML using the dedicated `MarkdownParser`.
-  - `_render_math`: Triggers MathJax rendering for newly added mathematical expressions.
-  - `_create_message_element`: Creates styled message elements with markdown support for chat display.
   - (Workspace operations like `save_workspace` are invoked via function calls processed by `ProcessFunctionCalls` which then use the client-side `WorkspaceManager` instance).
 
 ## Client-Side Brython Managers (in `static/client/managers/`)
 - A suite of manager classes, running in the Brython environment, handle the detailed logic for canvas objects and operations.
+- **`BaseDrawableManager`**: Base class providing shared infrastructure (container access, undo archiving, name generation) for all specialized drawable managers.
 - **`DrawableManager`**: Central orchestrator for all drawable objects. It coordinates the specialized managers listed below. Instantiated by `Canvas`.
-- **Specialized Drawable Managers**:
+- **Specialized Drawable Managers** (inherit from `BaseDrawableManager`):
   - `PointManager`: Manages point objects.
   - `SegmentManager`: Manages line segment objects.
   - `VectorManager`: Manages vector objects.
@@ -421,6 +434,7 @@ server_tests/
   - `DrawableDependencyManager`: Tracks dependencies between drawable objects.
   - `DrawableManagerProxy`: A proxy to help manage initialization dependencies for `DrawableManager`.
   - `DrawablesContainer`: A data structure used by `DrawableManager` to hold drawable instances.
+  - `VisibilityManager`: Controls drawable visibility state (extracted from `Canvas`).
 - These managers are responsible for creating, deleting, updating, and querying drawable objects on the canvas, and are used by `ProcessFunctionCalls` when executing drawing-related function calls.
 
 ## Workspace Management
@@ -532,7 +546,7 @@ server_tests/
           - Math evaluations (`ExpressionEvaluator`).
           - Client-side workspace operations (e.g., `save_workspace`) that call methods on the client `WorkspaceManager`, which then makes further AJAX calls.
       iii. Results from `ProcessFunctionCalls.get_results` are collected.
-      iv. `AIInterface._add_tool_call_entries` records each tool call in the tool call log dropdown, updating the live count in the UI.
+      iv. `ToolCallLogManager` records each tool call in the tool call log dropdown, updating the live count in the UI.
       v. `AIInterface._store_results_in_canvas_state` updates canvas computations.
       vi. `AIInterface._send_prompt_to_ai` is called again, this time with `tool_call_results` (and no `user_message`), looping back to step 4 (backend forwards tool results to OpenAI).
 8. Canvas is updated by the managers if drawing functions were called. Results are displayed in chat.

documentation/Reference Manual.txt

@@ -500,7 +500,7 @@ computation history for mathematical operations.
 - `_process_function_call(call, available_functions, non_computation_functions, unformattable_functions, canvas, results)`: Process a single function call and update results
 - `_is_function_available(function_name, available_functions, results)`: Check if the function exists and update results if not
 - `_execute_function(function_name, args, available_functions)`: Execute the function with the provided arguments
-- `_generate_result_key(function_name, args)`: Generate a consistent key format for the results dictionary
+- `generate_result_key(function_name, args)`: Generate a consistent key format for the results dictionary
 - `_process_result(function_name, args, result, key, unformattable_functions, non_computation_functions, canvas, results)`: Process the result based on function type
 - `_handle_exception(exception, function_name, results)`: Handle exceptions during function calls
 - `_add_computation_if_needed(result, function_name, non_computation_functions, expression, canvas)`: Add computation to canvas if needed