Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 1 addition & 19 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -61,25 +61,7 @@ updates:
- "automated"
- "security"

- package-ecosystem: "cargo"
directory: "/tools/wizer_initializer"
schedule:
interval: "weekly"
day: "monday"
time: "06:00"
open-pull-requests-limit: 10
reviewers:
- "avrabe"
assignees:
- "avrabe"
commit-message:
prefix: "rust/wizer"
include: "scope"
labels:
- "dependencies"
- "rust"
- "automated"
- "performance"
# Note: /tools/wizer_initializer removed - wizer is now part of wasmtime v39.0.0+

- package-ecosystem: "cargo"
directory: "/tools-builder/toolchains"
Expand Down
107 changes: 30 additions & 77 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,14 +75,13 @@ Before implementing any solution:

5. **Medium files IMPROVED** - 5 → 3 calls (-2)
- ✅ wkg_toolchain.bzl: cp → symlink
- ✅ wizer_toolchain.bzl: which → repository_ctx.which()
- ✅ wizer_toolchain.bzl: REMOVED (wizer now part of wasmtime v39.0.0+)

**REMAINING COMPLEX OPERATIONS (31 calls):**
**REMAINING COMPLEX OPERATIONS (29 calls):**

- **wasm_toolchain.bzl (17)**: Remaining download and build operations (hybrid approach working)
- **tool_cache.bzl (6)**: Tool validation and file existence checks
- **tinygo_toolchain.bzl (3)**: Tool installation and validation
- **wizer_toolchain.bzl (2)**: Script execution and version checking
- **Others (3)**: Package management and validation

**Shell Operation Categories MODERNIZED:**
Expand Down Expand Up @@ -155,94 +154,48 @@ These remaining shell scripts are **appropriate complexity** for their tasks:

## WIZER INTEGRATION STATUS

### 🎯 Complete Solution Architecture Implemented
### ✅ MIGRATION COMPLETE: Wasmtime v39.0.0+ Integration

**Problem**: Wizer CLI expects WebAssembly modules but WASI-enabled Rust toolchain produces components
**As of November 2025**, Wizer has been merged into Wasmtime and is available as the `wasmtime wizer` subcommand.
This eliminates the need for a standalone wizer toolchain and simplifies dependency management.

**Solution**: Library-based approach with component parsing
### Architecture

### ✅ COMPLETED COMPONENTS
The wizer pre-initialization workflow now uses wasmtime's built-in wizer subcommand:

1. **wizer_initializer Tool** (`//tools/wizer_initializer:wizer_initializer`)
- ✅ Bazel-native Rust binary with proper dependency management
- ✅ Component model detection (version 0x1000d vs 0x1)
- ✅ Architecture for component → module → wizer → component workflow
- ✅ Placeholder implementation demonstrating complete pipeline
- ✅ Full CLI interface with clap and anyhow
1. **wasm_component_wizer Rule** (`//wasm:wasm_component_wizer.bzl`)
- Uses `wasmtime_toolchain_type` instead of standalone wizer
- Invokes `wasmtime wizer` subcommand
- Default init function: `wizer-initialize` (breaking change from `wizer.initialize`)

2. **wasm_component_wizer_library Rule** (`//wasm:wasm_component_wizer_library.bzl`)
- ✅ Bazel rule using wizer_initializer for programmatic control
- ✅ Proper argument passing (--input, --output, --init-func, --allow-wasi, --verbose)
- ✅ Full integration with existing Bazel ecosystem
- ✅ Successfully tested with wizer_example
- Library-based pre-initialization support
- Uses wasmtime toolchain for wizer functionality

3. **Working Integration Test** (`//examples/wizer_example:wizer_library_test`)
- ✅ Successfully processes WebAssembly components (2.2MB test file)
- ✅ Correct component model detection and verbose logging
- ✅ Demonstrates complete architecture end-to-end
3. **Working Example** (`//examples/wizer_example`)
- Demonstrates pre-initialization with wasmtime wizer
- Uses `#[export_name = "wizer-initialize"]` (new naming convention)

### 🔧 CURRENT IMPLEMENTATION STATUS
### Breaking Changes (Issue #246)

**Working Foundation:**
- Init function name changed: `wizer.initialize` → `wizer-initialize`
- Standalone wizer toolchain removed (use wasmtime toolchain)
- `//tools/wizer_initializer` removed (no longer needed)
- `checksums/tools/wizer.json` removed

- ✅ Component/module format detection working perfectly
- ✅ Bazel rule integration working with proper error handling
- ✅ CLI argument processing and verbose logging working
- ✅ File I/O and Bazel integration working flawlessly

**Placeholder Components (for dependency resolution issues):**

- ⏳ Component parsing (requires wasm-tools or wasmtime integration)
- ⏳ Wizer library calls (requires wizer crate - complex dependencies)
- ⏳ Component wrapping (requires wasm-tools component new functionality)

### 🚀 ARCHITECTURE SUCCESS

The implemented solution **perfectly demonstrates** the correct approach:
### Migration Guide

**Before (standalone wizer):**
```rust
// Workflow: Component → Core Module → Wizer → Component
let is_component = is_wasm_component(&input_bytes)?; // ✅ Working
let core_module = extract_core_module(&input_bytes)?; // ⏳ Placeholder
let initialized = wizer.run(&core_module)?; // ⏳ Placeholder
let final_component = wrap_as_component(&initialized)?; // ⏳ Placeholder
#[export_name = "wizer.initialize"]
pub extern "C" fn init() { ... }
```

**Key Achievement**: The Bazel integration and component detection work perfectly. The remaining work is adding the
specific crate dependencies for:

1. `wasm-tools` for component parsing/wrapping
2. `wizer` crate for actual pre-initialization
3. `wasmtime` for runtime component support

### 🔬 COMPLEX DEPENDENCY ANALYSIS

**Issue**: Bazel crate_universe has build conflicts with Wizer/Wasmtime ecosystem:

- Cranelift (used by Wasmtime) has complex ISLE build system requirements
- Version conflicts between transitive dependencies
- Build script compatibility issues in sandboxed Bazel environment

**Alternative Approaches**:

1. **Shell out to system wizer** (against Bazel principles)
2. **Hermetic wasm-tools + wizer binaries** (current working approach with CLI)
3. **Library integration** (implemented architecture, requires dependency resolution)

### 📊 CURRENT STATE SUMMARY

| Component | Status | Notes |
| ---------------------- | -------------- | -------------------------------------- |
| Architecture Design | ✅ Complete | Library-based approach validated |
| Bazel Rule Integration | ✅ Complete | wasm_component_wizer_library working |
| Component Detection | ✅ Complete | Perfect WebAssembly format detection |
| CLI Tool Framework | ✅ Complete | Full argument processing and logging |
| Test Integration | ✅ Complete | Working end-to-end in wizer_example |
| Wizer Library Calls | ⏳ Placeholder | Requires complex dependency resolution |
| Component Parsing | ⏳ Placeholder | Requires wasm-tools or wasmtime crates |

**Bottom Line**: The architecture is complete and proven. The remaining work is purely dependency management for the
Wizer/Wasmtime ecosystem in Bazel.
**After (wasmtime wizer):**
```rust
#[export_name = "wizer-initialize"]
pub extern "C" fn init() { ... }
```

#### 📋 Implementation Guidelines

Expand Down
22 changes: 3 additions & 19 deletions MODULE.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -171,18 +171,8 @@ use_repo(tinygo, "tinygo_toolchain")

register_toolchains("@tinygo_toolchain//:tinygo_toolchain_def")

# Wizer WebAssembly pre-initialization toolchain
wizer = use_extension("//wasm:extensions.bzl", "wizer")
wizer.register(
name = "wizer",
strategy = "download",
version = "10.0.0",
)
use_repo(wizer, "wizer_toolchain")

register_toolchains("@wizer_toolchain//:wizer_toolchain_def")

# Wasmtime WebAssembly runtime
# Wasmtime WebAssembly runtime (includes wizer as of v39.0.0)
# Note: Standalone wizer toolchain removed - use `wasmtime wizer` subcommand instead
wasmtime = use_extension("//wasm:extensions.bzl", "wasmtime")
wasmtime.register(
name = "wasmtime",
Expand Down Expand Up @@ -262,11 +252,6 @@ register_toolchains("//toolchains:wasm_tools_component_toolchain_local")

# Rust crates for tools
crate = use_extension("@rules_rust//crate_universe:extension.bzl", "crate")
crate.from_cargo(
name = "wizer_crates",
cargo_lockfile = "//tools/wizer_initializer:Cargo.lock",
manifests = ["//tools/wizer_initializer:Cargo.toml"],
)
crate.from_cargo(
name = "crates",
cargo_lockfile = "//tools/checksum_updater:Cargo.lock",
Expand Down Expand Up @@ -325,7 +310,7 @@ crate.from_cargo(
# - wit_bindgen::rt module with correct allocator integration
# This replaces the previously embedded runtime stubs

use_repo(crate, "crates", "ssh_keygen_crates", "wasm_embed_aot_crates", "wasmsign2_crates", "wizer_crates")
use_repo(crate, "crates", "ssh_keygen_crates", "wasm_embed_aot_crates", "wasmsign2_crates")

# Modernized WASM tool repositories using git_repository + rules_rust
wasm_tool_repos = use_extension("//toolchains:extensions.bzl", "wasm_tool_repositories")
Expand All @@ -335,6 +320,5 @@ use_repo(
"wasm_tools_src",
"wasmsign2_src",
"wit_bindgen_src",
"wizer_src",
"wrpc_src",
)
70 changes: 2 additions & 68 deletions checksums/registry.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -781,73 +781,7 @@ def _get_fallback_checksums(tool_name):
},
},
},
"wizer": {
"tool_name": "wizer",
"github_repo": "bytecodealliance/wizer",
"latest_version": "10.0.0",
"versions": {
"10.0.0": {
"release_date": "2025-08-27",
"platforms": {
"darwin_amd64": {
"sha256": "d6b2073efb38624399c1c6fc0d8c1e19ba24d7ab934f51c346b7d97d4bd8115c",
"url_suffix": "x86_64-macos.tar.xz",
"strip_prefix": "wizer-v10.0.0-x86_64-macos",
},
"darwin_arm64": {
"sha256": "5e3977e25defc869141dd8df3d25dde1dabe37ea377b7c01fa24693fe5bc7b3f",
"url_suffix": "aarch64-macos.tar.xz",
"strip_prefix": "wizer-v10.0.0-aarch64-macos",
},
"linux_amd64": {
"sha256": "1e9dfaa2f6c436ca7acfdbb82c166f5b8a0062068f12916a554e82c97ca8d371",
"url_suffix": "x86_64-linux.tar.xz",
"strip_prefix": "wizer-v10.0.0-x86_64-linux",
},
"linux_arm64": {
"sha256": "8ec7e50622aa7af37b5152f19d7a371f185c7af138033759caa3fdd3e17e2830",
"url_suffix": "aarch64-linux.tar.xz",
"strip_prefix": "wizer-v10.0.0-aarch64-linux",
},
"windows_amd64": {
"sha256": "29a3aebb70927e9af7c505d3f3d14c5f8577eb307e8bd45151c184885a580d2f",
"url_suffix": "x86_64-windows.zip",
"strip_prefix": "wizer-v10.0.0-x86_64-windows",
},
},
},
"9.0.0": {
"release_date": "2024-06-03",
"platforms": {
"darwin_amd64": {
"sha256": "5d5e457abf3fd6e307dee9fe9f7423185a88d90f0c96677b9a5418c448ced52e",
"url_suffix": "x86_64-macos.tar.xz",
"strip_prefix": "wizer-v9.0.0-x86_64-macos",
},
"darwin_arm64": {
"sha256": "3372ee8215abc39b15a51b4aed27f8ae5a42e84261a29e7491ec82bf806bc491",
"url_suffix": "aarch64-macos.tar.xz",
"strip_prefix": "wizer-v9.0.0-aarch64-macos",
},
"linux_amd64": {
"sha256": "d1d85703bc40f18535e673992bef723dc3f84e074bcd1e05b57f24d5adb4f058",
"url_suffix": "x86_64-linux.tar.xz",
"strip_prefix": "wizer-v9.0.0-x86_64-linux",
},
"linux_arm64": {
"sha256": "f560a675d686d42c18de8bd4014a34a0e8b95dafbd696bf8d54817311ae87a4d",
"url_suffix": "aarch64-linux.tar.xz",
"strip_prefix": "wizer-v9.0.0-aarch64-linux",
},
"windows_amd64": {
"sha256": "d9cc5ed028ca873f40adcac513812970d34dd08cec4397ffc5a47d4acee8e782",
"url_suffix": "x86_64-windows.zip",
"strip_prefix": "wizer-v9.0.0-x86_64-windows",
},
},
},
},
},
# Note: wizer removed - now part of wasmtime v39.0.0+, use `wasmtime wizer` subcommand
"jco": {
"tool_name": "jco",
"github_repo": "bytecodealliance/jco",
Expand Down Expand Up @@ -1013,6 +947,7 @@ def list_available_tools():
"""

# Return tools that have fallback data available
# Note: wizer removed - now part of wasmtime v39.0.0+, use `wasmtime wizer` subcommand
return [
"wasm-tools",
"wit-bindgen",
Expand All @@ -1021,7 +956,6 @@ def list_available_tools():
"wasmtime",
"wasi-sdk",
"wasmsign2",
"wizer",
"nodejs",
"jco",
"file-ops-component",
Expand Down
34 changes: 17 additions & 17 deletions checksums/registry_test.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,9 @@ def _test_get_tool_checksum(ctx):
checksum = get_tool_checksum("wasm-tools", "1.235.0", "darwin_amd64")
asserts.equals(env, "154e9ea5f5477aa57466cfb10e44bc62ef537e32bf13d1c35ceb4fedd9921510", checksum)

# Test wizer checksum (our new addition)
wizer_checksum = get_tool_checksum("wizer", "9.0.0", "linux_amd64")
asserts.equals(env, "d1d85703bc40f18535e673992bef723dc3f84e074bcd1e05b57f24d5adb4f058", wizer_checksum)
# Test wasmtime checksum (wizer functionality now included in wasmtime v39.0.0+)
wasmtime_checksum = get_tool_checksum("wasmtime", "39.0.1", "linux_amd64")
asserts.equals(env, "b90a36125387b75db59a67a1c402f2ed9d120fa43670d218a559571e2423d925", wasmtime_checksum)

# Test invalid tool
invalid_checksum = get_tool_checksum("nonexistent-tool", "1.0.0", "linux_amd64")
Expand All @@ -48,11 +48,11 @@ def _test_get_tool_info(ctx):
asserts.equals(env, "154e9ea5f5477aa57466cfb10e44bc62ef537e32bf13d1c35ceb4fedd9921510", info["sha256"])
asserts.equals(env, "x86_64-macos.tar.gz", info["url_suffix"])

# Test wizer info
wizer_info = get_tool_info("wizer", "9.0.0", "windows_amd64")
asserts.true(env, wizer_info != None)
asserts.equals(env, "d9cc5ed028ca873f40adcac513812970d34dd08cec4397ffc5a47d4acee8e782", wizer_info["sha256"])
asserts.equals(env, "x86_64-windows.zip", wizer_info["url_suffix"])
# Test wasmtime info (wizer functionality now included in wasmtime v39.0.0+)
wasmtime_info = get_tool_info("wasmtime", "39.0.1", "windows_amd64")
asserts.true(env, wasmtime_info != None)
asserts.equals(env, "bccf64b4227d178c0d13f2856be68876eae3f2f657f3a85d46f076a5e1976198", wasmtime_info["sha256"])
asserts.equals(env, "x86_64-windows.zip", wasmtime_info["url_suffix"])

return unittest.end(env)

Expand All @@ -79,12 +79,12 @@ def _test_list_supported_platforms(ctx):
asserts.true(env, "linux_amd64" in platforms)
asserts.true(env, "windows_amd64" in platforms)

# Test wizer platforms
wizer_platforms = list_supported_platforms("wizer", "9.0.0")
asserts.true(env, "darwin_amd64" in wizer_platforms)
asserts.true(env, "linux_amd64" in wizer_platforms)
asserts.true(env, "windows_amd64" in wizer_platforms)
asserts.equals(env, 5, len(wizer_platforms)) # Should have 5 platforms
# Test wasmtime platforms (wizer functionality now included in wasmtime v39.0.0+)
wasmtime_platforms = list_supported_platforms("wasmtime", "39.0.1")
asserts.true(env, "darwin_amd64" in wasmtime_platforms)
asserts.true(env, "linux_amd64" in wasmtime_platforms)
asserts.true(env, "windows_amd64" in wasmtime_platforms)
asserts.equals(env, 5, len(wasmtime_platforms)) # Should have 5 platforms

return unittest.end(env)

Expand All @@ -95,7 +95,7 @@ def _test_get_github_repo(ctx):
# Test known repos
asserts.equals(env, "bytecodealliance/wasm-tools", get_github_repo("wasm-tools"))
asserts.equals(env, "bytecodealliance/wit-bindgen", get_github_repo("wit-bindgen"))
asserts.equals(env, "bytecodealliance/wizer", get_github_repo("wizer"))
asserts.equals(env, "bytecodealliance/wasmtime", get_github_repo("wasmtime"))

# Test invalid tool
asserts.equals(env, None, get_github_repo("nonexistent-tool"))
Expand All @@ -108,7 +108,7 @@ def _test_validate_tool_exists(ctx):

# Test valid combinations
asserts.true(env, validate_tool_exists("wasm-tools", "1.235.0", "darwin_amd64"))
asserts.true(env, validate_tool_exists("wizer", "9.0.0", "linux_amd64"))
asserts.true(env, validate_tool_exists("wasmtime", "39.0.1", "linux_amd64"))

# Test invalid combinations
asserts.false(env, validate_tool_exists("nonexistent-tool", "1.0.0", "linux_amd64"))
Expand All @@ -124,7 +124,7 @@ def _test_list_available_tools(ctx):
tools = list_available_tools()
asserts.true(env, "wasm-tools" in tools)
asserts.true(env, "wit-bindgen" in tools)
asserts.true(env, "wizer" in tools)
asserts.true(env, "wasmtime" in tools) # wizer functionality now in wasmtime
asserts.true(env, "wac" in tools)
asserts.true(env, "wkg" in tools)

Expand Down
Loading