From 89c733994d1985f8ecdd7d998469abe44469ad8c Mon Sep 17 00:00:00 2001 From: mikebender Date: Mon, 19 Jan 2026 10:22:57 -0500 Subject: [PATCH 1/3] docs: add AGENTS.md guide for AI agents Add comprehensive guide for AI agents working in the deephaven-plugins repository covering: - Environment setup (Python and JavaScript) - Building plugins (Python wheels and JS bundles) - Running unit tests (tox) and E2E tests (Playwright) - Running development server (deephaven-core and Docker) - Documentation preview and snapshot generation - Code quality and pre-commit hooks - Common development workflows - Release management Prioritizes local development workflows over Docker-based approaches. --- AGENTS.md | 280 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 280 insertions(+) create mode 100644 AGENTS.md diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 000000000..f6eaf6c99 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,280 @@ +# AI Agent Guide for deephaven-plugins + +This guide provides essential information for AI agents working in the deephaven-plugins repository. + +## Repository Overview + +This repository contains Deephaven plugin modules, including both Python and JavaScript plugins located in the `plugins/` folder. + +## Environment Setup + +### Python Environment + +```bash +# Create and activate virtual environment +python -m venv .venv +source .venv/bin/activate + +# Install dependencies +pip install --upgrade -r requirements.txt + +# Install pre-commit hooks +pre-commit install +``` + +### JavaScript Environment + +```bash +# Install JS dependencies +npm install +``` + +## Building Plugins + +### Building Python Plugins + +Build Python plugin wheels from the root directory: + +```bash +# Build specific plugins +python -m build --wheel plugins/matplotlib +python -m build --wheel plugins/plotly + +# Install built wheels +pip install plugins/*/dist/*.whl + +# Force reinstall during development (without version bump) +pip install --force-reinstall --no-deps plugins/*/dist/*.whl +``` + +### Building JavaScript Plugins + +```bash +# Build all JS plugins in watch mode (also serves plugins directory) +npm start + +# Build specific plugins matching a pattern +npm start -- --scope *theme* + +# Build a single plugin in watch mode (without dev server) +cd plugins/ +npm start +``` + +The dev server runs at `http://localhost:4100` (configurable via `PORT` env variable). + +### Using plugin_builder.py + +The `plugin_builder.py` script automates common development tasks: + +```bash +# Install click and watchdog dependencies +pip install click watchdog + +# Configure environment (minimal) +python tools/plugin_builder.py --configure=min + +# Configure environment (full - includes docs and server) +python tools/plugin_builder.py --configure=full + +# Build all plugins +python tools/plugin_builder.py + +# Build specific plugins +python tools/plugin_builder.py plotly-express ui + +# Build with JS plugins +python tools/plugin_builder.py --js ui + +# Install/reinstall plugins +python tools/plugin_builder.py --install plotly-express +python tools/plugin_builder.py --reinstall ui + +# Build docs for a plugin +python tools/plugin_builder.py --docs --install ui + +# Re-generate doc snapshots +python tools/plugin_builder.py --docs --snapshots ui + +# Start server with plugins +python tools/plugin_builder.py --reinstall --server plotly-express + +# Watch mode (rebuilds on changes) +python tools/plugin_builder.py --watch plotly-express + +# Combined flags (short form) +python tools/plugin_builder.py -jrsw ui # js, reinstall, server, watch +``` + +## Running Tests + +### Unit Tests (Python) + +Python plugins use `tox` for testing: + +```bash +# Run tests for a plugin (from plugins/ directory) +cd plugins/ +tox -e py + +# Run tests against specific Python version +tox -e py3.12 +``` + +Note: `tox` runs against Python 3.9 by default. Ensure the target Python version is installed on your system. + +### End-to-End Tests + +E2E tests use Playwright and test against Chrome, Firefox, and Webkit: + +```bash +# Run tests locally in UI mode (recommended for development/debugging) +npm run e2e:ui + +# Run E2E tests in Docker (matches CI environment, use for validation) +npm run e2e:docker + +# Run specific test file +npm run e2e:docker -- ./tests/matplotlib.spec.ts + +# Run specific test in specific browser +npm run e2e:docker -- --project firefox ./tests/matplotlib.spec.ts + +# Update snapshots (use Docker for consistency with CI) +npm run e2e:update-snapshots + +# Update snapshots for specific test +npm run e2e:update-snapshots -- ./tests/ui.spec.ts +``` + +**Note**: If you encounter permissions issues after running Docker tests, delete the test-results folder: `sudo rm -rf test-results` + +## Running the Development Server + +### With deephaven-core (Recommended) + +1. Build deephaven-core following [these instructions](https://deephaven.io/core/docs/getting-started/launch-build/#build-and-run-deephaven) + +2. Install plugin wheels into deephaven-core venv: + +```bash +pip install /plugins/*/dist/*.whl +``` + +3. Start deephaven-core with js-plugin flags: + +```bash +START_OPTS="-Ddeephaven.jsPlugins.@deephaven/js-plugin-matplotlib=/plugins/matplotlib/src/js -Ddeephaven.jsPlugins.@deephaven/js-plugin-plotly=/plugins/plotly/src/js" ./gradlew server-jetty-app:run +``` + +Server opens at `http://localhost:10000/ide/` + +### With Docker (Alternative) + +```bash +# Start Docker container with all plugins +npm run docker +# or +docker compose up --build + +# Use custom port +DEEPHAVEN_PORT=11000 npm run docker +``` + +Server runs at `http://localhost:10000` by default. JS plugins are configured in `./docker/config/deephaven.prop`. + +## Documentation + +### Preview Docs + +```bash +# Preview docs from source (without API reference) +npm run docs + +# Build and preview full docs +python tools/plugin_builder.py -d ui plotly-express +BUILT=true npm run docs +``` + +Docs preview runs at `http://localhost:3001`. + +### Update Doc Snapshots + +```bash +# Using plugin_builder.py +python tools/plugin_builder.py --docs --snapshots ui + +# Using npm +npm run update-doc-snapshots +``` + +## Code Quality + +### Pre-commit Hooks + +The repository uses pre-commit hooks for: + +- Black and blacken-docs formatting +- Pyright type checking +- Ruff linting + +```bash +# Test pre-commit hooks +pre-commit run --all-files + +# Bypass pre-commit on commit +git commit --no-verify -m "commit message" +``` + +## Common Development Workflows + +### Developing a Python Plugin + +```bash +python tools/plugin_builder.py --reinstall +cd plugins/ +tox -e py +``` + +### Developing a JS Plugin + +```bash +npm start -- --scope ** +npm run e2e:ui # for testing +``` + +### Full Development Cycle with Server + +```bash +# Build everything and start server with watch mode +python tools/plugin_builder.py -jrsw +``` + +### Running All Tests + +```bash +# Python unit tests +cd plugins/ +tox -e py + +# E2E tests (local UI mode for development) +npm run e2e:ui + +# E2E tests (Docker for CI validation) +npm run e2e:docker +``` + +## Release Management + +See the main README.md for detailed release procedures using `cocogitto` (`cog`). + +```bash +# Release a plugin +tools/release.sh +``` + +This must be done on the `main` branch and requires: + +- GitHub CLI tool +- cocogitto (cog) installed +- Conventional commit messages From 793120b0982c8cb9f1959dc883445c9952f499bb Mon Sep 17 00:00:00 2001 From: mikebender Date: Mon, 19 Jan 2026 10:27:19 -0500 Subject: [PATCH 2/3] docs: add feature planning guidelines for plugins Add feature planning infrastructure: - Create plugins/ui/plans/ directory for feature plan documents - Add README with naming convention (ticket number + keyword) - Define plan document structure (overview, goals, design, implementation, testing, docs, open questions) - Update AGENTS.md with general instructions for creating feature plans in any plugin --- AGENTS.md | 10 ++++++++++ plugins/ui/plans/README.md | 33 +++++++++++++++++++++++++++++++++ 2 files changed, 43 insertions(+) create mode 100644 plugins/ui/plans/README.md diff --git a/AGENTS.md b/AGENTS.md index f6eaf6c99..fac0d6e96 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -6,6 +6,16 @@ This guide provides essential information for AI agents working in the deephaven This repository contains Deephaven plugin modules, including both Python and JavaScript plugins located in the `plugins/` folder. +## Feature Planning + +For complex features, create detailed plan documents in `plugins//plans/`: + +- **Naming**: Use ticket numbers (e.g., `DH-12345.md`) +- **Structure**: Include overview, goals, technical design, implementation plan, testing strategy, documentation needs, and open questions +- **Purpose**: Design documentation, implementation reference, and historical record of decisions + +See [plugins/ui/plans/README.md](plugins/ui/plans/README.md) for complete guidelines. + ## Environment Setup ### Python Environment diff --git a/plugins/ui/plans/README.md b/plugins/ui/plans/README.md new file mode 100644 index 000000000..0c4739696 --- /dev/null +++ b/plugins/ui/plans/README.md @@ -0,0 +1,33 @@ +# UI Plugin Feature Plans + +This directory contains detailed feature plans and design documents for the deephaven.ui plugin. + +## Naming Convention + +Each plan document should be named using its corresponding ticket number followed by a human-readable keyword: + +``` +DH-12345 my-feature-name.md +DH-67890 another-feature.md +``` + +## Plan Document Structure + +Each plan document should include: + +1. **Overview** - Brief description of the feature +2. **Goals** - What the feature aims to accomplish +3. **Technical Design** - Implementation details, API design, architecture +4. **Implementation Plan** - Step-by-step breakdown of tasks +5. **Testing Strategy** - Unit tests, E2E tests, edge cases +6. **Documentation** - What docs need to be created or updated +7. **Open Questions** - Any unresolved design decisions + +## Purpose + +These plans serve as: + +- Design documentation for complex features +- Reference for implementation +- Historical record of design decisions +- Communication tool for stakeholders and AI agents From f1f3d2922af1d3f7b2901a6b565bc050c0f16271 Mon Sep 17 00:00:00 2001 From: mikebender Date: Mon, 19 Jan 2026 10:42:12 -0500 Subject: [PATCH 3/3] feat: add feature plan for DH-21375 databars integration with TableFormat Create comprehensive plan document for integrating databars into ui.TableFormat: - Align databars with other formatting features for API consistency - Enable conditional databars with if_ parameter support - Maintain backward compatibility with deprecated databars parameter - Define 5-phase implementation: foundation, JS integration, testing, docs, cleanup - Document testing strategy, migration guide, and success criteria - Address open design questions with recommendations --- .../ui/plans/DH-21375 databars-tableformat.md | 251 ++++++++++++++++++ 1 file changed, 251 insertions(+) create mode 100644 plugins/ui/plans/DH-21375 databars-tableformat.md diff --git a/plugins/ui/plans/DH-21375 databars-tableformat.md b/plugins/ui/plans/DH-21375 databars-tableformat.md new file mode 100644 index 000000000..e32102cdb --- /dev/null +++ b/plugins/ui/plans/DH-21375 databars-tableformat.md @@ -0,0 +1,251 @@ +# DH-21375 Databars need to be integrated with `ui.TableFormat` + +## Overview + +Currently, databars on tables are configured through the `databars` parameter of `ui.table()`, which is separate from the general table formatting API. This creates inconsistency in the API design, as all other formatting features (colors, alignment, value formatting, etc.) are handled through `ui.TableFormat` objects passed to the `format_` parameter. + +This feature will integrate databars into the `ui.TableFormat` construct to provide a consistent, unified formatting API. + +## Goals + +1. **API Consistency**: Align databars with other formatting features by making them part of `ui.TableFormat` +2. **Simplified Mental Model**: Users learn one formatting system instead of two separate APIs +3. **Feature Parity**: Maintain all existing databar functionality while improving the API +4. **Backward Compatibility**: Deprecate the old `databars` parameter gracefully without breaking existing code +5. **Enhanced Flexibility**: Enable databars to work with conditional formatting rules (`if_` parameter) + +## Technical Design + +### Current Implementation + +**Python API (table.py):** + +- `databars` parameter accepts `list[dict]` with databar configs +- Separate `TableDatabar` dataclass exists but isn't fully integrated +- No connection to `TableFormat` system + +**JavaScript Processing (UITableModel.ts):** + +- `databars` array processed separately from `format` rules +- Joins table with min/max totals for dynamic scaling +- Creates databar configs mapped by column name + +### Proposed Changes + +#### 1. Python API Changes + +**Update `TableFormat` to fully support databars:** + +```python +@dataclass +class TableFormat: + cols: ColumnName | list[ColumnName] | None = None + if_: str | None = None + color: Color | None = None + background_color: Color | None = None + alignment: Literal["left", "center", "right"] | None = None + value: str | None = None + mode: Literal["databar"] | None = None # Change from TableDatabar to literal + + # Databar-specific properties (only used when mode="databar") + value_column: ColumnName | None = None + min: ColumnName | float | None = None + max: ColumnName | float | None = None + axis: Literal["proportional", "middle", "directional"] | None = None + direction: Literal["LTR", "RTL"] | None = None + value_placement: Literal["beside", "overlap", "hide"] | None = None + opacity: float | None = None + markers: list[dict] | None = None # For marker lines +``` + +**Example usage:** + +```python +t = ui.table( + dx.data.stocks(), + format_=[ + # Simple databar + ui.TableFormat(cols="Price", mode="databar", color="positive"), + # Databar with conditional formatting + ui.TableFormat( + cols="Size", + if_="Size > 100", + mode="databar", + color=["negative", "positive"], + min=0, + max=1000, + ), + # Other formatting still works + ui.TableFormat(cols="Sym", background_color="accent-100"), + ], +) +``` + +**Deprecate `databars` parameter:** + +- Keep `databars` parameter functional for backward compatibility +- Add deprecation warning when used +- Document migration path in release notes + +#### 2. JavaScript Changes + +**UITableUtils.ts:** + +- Update `FormattingRule` interface to include databar properties +- Add type guards to identify databar formatting rules + +**UITableModel.ts:** + +- Merge databar processing with format rule processing +- Extract databar configs from format rules where `mode === "databar"` +- Maintain existing databar rendering logic +- Support legacy `databars` parameter during deprecation period + +**UITable.tsx:** + +- Process both `databars` prop (deprecated) and format rules with databars +- Show console warning when `databars` prop is used +- Pass combined databar configs to UITableModel + +#### 3. Documentation Changes + +**Update table.md:** + +- Move databar documentation from separate section into formatting section +- Show examples using `ui.TableFormat` with `mode="databar"` +- Add migration guide from old to new API +- Keep warning about API changes but update to point to new approach + +**Update API reference:** + +- Document new `mode` parameter and databar properties +- Mark `databars` parameter as deprecated +- Provide clear examples of both approaches during transition + +## Implementation Plan + +### Phase 1: Foundation + +1. Update `TableFormat` dataclass with databar properties +2. Add type definitions and validation +3. Write unit tests for new TableFormat API + +### Phase 2: JavaScript Integration + +4. Update TypeScript interfaces in UITableUtils.ts +5. Modify UITableModel.ts to process databars from format rules +6. Add support for both old and new APIs simultaneously +7. Add console warning for deprecated `databars` parameter + +### Phase 3: Testing + +8. Create E2E tests for new databar format API +9. Verify backward compatibility with existing databars tests +10. Test conditional databars (`if_` parameter) +11. Test databar priority with multiple format rules + +### Phase 4: Documentation + +12. Update component documentation +13. Create migration guide +14. Update code examples throughout docs +15. Update CHANGELOG with deprecation notice + +### Phase 5: Cleanup (Future Release) + +16. Remove deprecated `databars` parameter (after 2-3 releases) +17. Remove legacy code paths +18. Simplify internal implementation + +## Testing Strategy + +### Unit Tests (Python) + +- `TableFormat` with `mode="databar"` validates correctly +- Databar properties are serialized properly +- Invalid configurations raise appropriate errors +- Mixing databars with other format properties works + +### E2E Tests (Playwright) + +- Databars render correctly using new `TableFormat` API +- Conditional databars work with `if_` parameter +- All databar features functional (colors, gradients, markers, opacity, etc.) +- Format rule priority applies to databars +- Backward compatibility: old `databars` parameter still works +- Console warning appears when using deprecated API + +### Manual Testing + +- Visual verification of databar rendering +- Interactive testing of dynamic databars with ticking tables +- Performance testing with large tables and many format rules + +## Documentation + +### Updates Needed + +1. **plugins/ui/docs/components/table.md** + + - Move databar section under "Formatting" + - Update all examples to use `ui.TableFormat` + - Add migration guide from old API + - Keep brief note about deprecated `databars` param + +2. **API Reference** + + - Document new `mode` parameter + - Document databar-specific properties on `TableFormat` + - Mark `databars` parameter as deprecated + +3. **CHANGELOG.md** + + - Note deprecation of `databars` parameter + - Explain migration path + - List new capabilities (conditional databars) + +4. **Migration Guide** + + ```python + # Old API (deprecated) + ui.table(t, databars=[{"column": "Price", "color": "positive"}]) + + # New API + ui.table(t, format_=[ui.TableFormat(cols="Price", mode="databar", color="positive")]) + ``` + +## Open Questions + +1. **Should we keep `TableDatabar` dataclass?** + + - Option A: Remove it entirely, flatten all properties into `TableFormat` + - Option B: Keep it as `mode=TableDatabar(...)` for grouping databar options + - **Recommendation**: Option A (flatten) for simplicity and consistency + +2. **How long should we support the deprecated `databars` parameter?** + + - **Recommendation**: 3-4 releases (~3-6 months) with deprecation warnings + +3. **Should databar-specific properties be valid when `mode != "databar"`?** + + - **Recommendation**: Ignore them (don't raise errors) for forward compatibility + +4. **Can multiple format rules create databars on the same column?** + + - **Recommendation**: Last rule wins (consistent with other format priority) + +5. **Should we support databars without specifying columns (row-level databars)?** + + - **Recommendation**: No, databars are inherently column-specific + +6. **How do databars interact with other `mode` values in the future?** + - **Recommendation**: `mode` is mutually exclusive; last rule wins + +## Success Criteria + +- [ ] All existing databar functionality works through `ui.TableFormat` +- [ ] Backward compatibility maintained with deprecation warnings +- [ ] E2E tests pass for both old and new APIs +- [ ] Documentation fully updated with examples +- [ ] Performance equivalent or better than current implementation +- [ ] User feedback is positive during beta period