implementation of AWS S3 Tables / Apache Iceberg catalog support

Cargo.toml

@@ -38,7 +38,7 @@ async-recursion = "1.1"
 async-stream = "0.3"
 async-trait = "0.1"
 base64 = "0.22"
-chrono = "0.4"
+chrono = { version = "0.4", features = ["serde"] }
 crc = "3.3"
 dashmap = "6.1.0"
 env_logger = "0.11"

DEBUGGING_SESSION_2.md

@@ -0,0 +1,107 @@
+# Tables API Signature Debugging - Session 2
+
+## Investigation Summary
+
+### Key Findings
+
+1. **Signature Calculation is Correct**
+   - Service name: `s3tables` ✓
+   - Region: `us-east-1` ✓
+   - Content SHA256 matches between header and signing ✓
+   - All required headers present before signing ✓
+   - Authorization header format correct ✓
+
+2. **Server Authentication Flow Verified**
+   - Server uses `serviceTables = "s3tables"` (cmd/signature-v4.go:45)
+   - Server trusts client-provided x-amz-content-sha256 header for s3tables service (cmd/signature-v4-utils.go:102-119)
+   - Server expects standard AWS Signature V4 format
+
+3. **SDK Implementation Matches Server Tests**
+   - Server test file (cmd/test-utils_test.go:793-892) shows correct signing process
+   - Our SDK follows the same pattern
+   - SHA256 calculations match: header value == signing value
+
+### Debug Output From Last Test Run
+
+```
+[execute_tables] Body SHA256 (for header): 4af03460a4c315ffbaf74aaa140180b82315019f49d5985d8f629b9a5137416a
+[execute_tables] Body length: 57
+[execute_tables] Added X-Amz-Content-SHA256 header: 4af03460a4c315ffbaf74aaa140180b82315019f49d5985d8f629b9a5137416a
+[execute_tables] Region: 'us-east-1'
+[execute_tables] Access Key: henk
+[execute_tables] Headers BEFORE signing:
+  X-Amz-Content-SHA256: 4af03460a4c315ffbaf74aaa140180b82315019f49d5985d8f629b9a5137416a
+  Host: localhost:9000
+  Content-Length: 57
+  X-Amz-Date: 20251023T090439Z
+  Content-Type: application/json
+[sign_v4_s3tables] Body SHA256 (for signing): 4af03460a4c315ffbaf74aaa140180b82315019f49d5985d8f629b9a5137416a
+[sign_v4_s3tables] Body length: 57
+[execute_tables] URL: http://localhost:9000/_iceberg/v1/warehouses
+[execute_tables] All headers:
+  Authorization: AWS4-HMAC-SHA256 Credential=henk/20251023/us-east-1/s3tables/aws4_request, SignedHeaders=content-length;content-type;host;x-amz-content-sha256;x-amz-date, Signature=a84006c6b9966cbfe6c304a11f21748768bd9843871f8abdf7fdf2bbe8323c89
+  Content-Type: application/json
+  Host: localhost:9000
+  X-Amz-Content-SHA256: 4af03460a4c315ffbaf74aaa140180b82315019f49d5985d8f629b9a5137416a
+  X-Amz-Date: 20251023T090439Z
+  Content-Length: 57
+```
+
+**Error**: `TablesError(Generic("The request signature we calculated does not match the signature you provided. Check your key and signing method."))`
+
+### Files Modified with Debug Logging
+
+1. `src/s3/client.rs` - execute_tables function (lines 635-679)
+2. `src/s3/signer.rs` - sign_v4_s3tables function (lines 179-186)
+
+### Potential Issues to Investigate
+
+1. **Server Branch**: Currently on `tp/register-table` - verify this branch has complete Tables API authentication implementation
+2. **Credentials**: Tested with both `henk/$MINIO_ROOT_PASSWORD` and `minioadmin/minioadmin` - both fail
+3. **URL Encoding**: Server uses `s3utils.EncodePath()` - need to verify our URI matches (currently using `/_iceberg/v1/warehouses` without encoding)
+4. **Header Canonicalization**: Verify multimap produces headers in exact format server expects
+5. **Time Sync**: Minor - 26 second difference between request and server time should not cause issues
+
+### Next Steps
+
+1. **Enable server debug logging** to see what canonical request the server is calculating
+   - Compare server's canonical request with SDK's
+   - Check if there's a mismatch in header ordering, URI encoding, or query string format
+
+2. **Create minimal reproduction** using curl with manual AWS SigV4 signing to isolate SDK vs server issue
+
+3. **Verify server configuration**:
+   - Check if server is using correct credentials for user `henk`
+   - Verify server region configuration matches `us-east-1`
+   - Confirm branch `tp/register-table` has Tables API fully implemented
+
+4. **Check for middleware** that might modify requests between client and authentication handler
+
+### Test Command
+
+```bash
+SERVER_ENDPOINT="http://localhost:9000/" ENABLE_HTTPS="false" ACCESS_KEY="henk" SECRET_KEY="$MINIO_ROOT_PASSWORD" cargo test --test test_tables_create_delete warehouse_create -- --nocapture
+```
+
+### Server Information
+
+- Binary: `C:\minio\minio.exe`
+- Branch: `tp/register-table`
+- Uptime: 17+ hours
+- Tables API endpoint: `/_iceberg/v1`
+- Service type: `serviceTables = "s3tables"`
+
+## Conclusion
+
+The SDK implementation appears correct based on:
+- Matching server test implementation
+- Correct AWS SigV4 format
+- All required headers present
+- Matching SHA256 calculations
+
+The issue likely lies in:
+- Server-side configuration
+- Branch-specific authentication changes not documented
+- Subtle difference in canonical request construction (URI encoding, header ordering, etc.)
+
+**Recommendation**: Enable server-side debug logging or add logging to `cmd/signature-v4.go:doesSignatureMatch()` function to print the server's calculated canonical request and compare with SDK's output.

DEBUGGING_SESSION_3.md

@@ -0,0 +1,195 @@
+# Tables API Signature Debugging - Session 3
+
+## Session Summary
+
+Attempted to add server-side debug logging to compare client and server canonical request calculations, but encountered persistent Go build issues on Windows.
+
+## SDK Enhancements Made
+
+### Added Canonical Request Debug Output
+
+Modified `src/s3/signer.rs:get_canonical_request_hash()` to print detailed canonical request construction (lines 71-80):
+
+```rust
+eprintln!("\n=== CANONICAL REQUEST DEBUG ===");
+eprintln!("Method: {}", method);
+eprintln!("URI: {}", uri);
+eprintln!("Query String: '{}'", query_string);
+eprintln!("Headers:\n{}", headers);
+eprintln!("Signed Headers: {}", signed_headers);
+eprintln!("Content SHA256: {}", content_sha256);
+eprintln!("\nFull Canonical Request:");
+eprintln!("{}", canonical_request);
+eprintln!("=== END CANONICAL REQUEST ===\n");
+```
+
+### Canonical Request Output
+
+Test run shows the SDK generates the following canonical request for CreateWarehouse:
+
+```
+POST
+/_iceberg/v1/warehouses
+
+content-length:57
+content-type:application/json
+host:localhost:9000
+x-amz-content-sha256:dd76107cb09a4c9862be38e9487a3c99f8bbb230994040c14805995cddcd5204
+x-amz-date:20251023T095353Z
+
+content-length;content-type;host;x-amz-content-sha256;x-amz-date
+dd76107cb09a4c9862be38e9487a3c99f8bbb230994040c14805995cddcd5204
+```
+
+**Analysis**: This canonical request format is **correct** according to AWS Signature Version 4 specification:
+- ✅ HTTP method on first line
+- ✅ Canonical URI on second line
+- ✅ Empty query string on third line
+- ✅ Canonical headers (lowercase, sorted, format `key:value`)
+- ✅ Blank line separator
+- ✅ Signed headers list (semicolon-separated)
+- ✅ Payload hash
+
+## Server-Side Debug Logging Attempts
+
+### Files Modified
+
+1. **cmd/signature-v4.go** (lines 382-415)
+   - Added debug output in `doesSignatureMatch()` for `serviceTables`
+   - Prints: service, method, path, region, hashed payload, query string, signed headers, canonical request, scope, string to sign, calculated vs provided signatures
+
+2. **cmd/auth-handler.go** (lines 716-739)
+   - Added debug output in `reqSignatureV4Verify()` for `serviceTables`
+   - Prints: request method/path, service type, region, SHA256 sum
+
+3. **cmd/tables-api-handlers.go** (lines 89-107)
+   - Added debug output in `CreateWarehouse()` handler
+   - Prints: request method/path, authorization header, auth check results
+
+All files had proper imports added (`fmt`, `os`).
+
+### Go Build Issues on Windows
+
+**Problem**: Every `go build` command produces an archive file instead of a Windows PE executable:
+
+```bash
+$ file minio.exe
+minio.exe: current ar archive  # WRONG - should be "PE32+ executable"
+```
+
+**Attempted Solutions** (all failed):
+1. `go build -o /c/minio/minio.exe ./cmd` → archive
+2. `env CGO_ENABLED=0 GOOS=windows GOARCH=amd64 go build -trimpath -o /c/minio/minio-debug.exe ./cmd` → archive
+3. `go install -trimpath -a ./cmd` → archive
+4. Build from cmd directory directly → archive
+
+**Error When Trying to Execute**:
+```
+./minio.exe: line 1: syntax error near unexpected token `newline'
+./minio.exe: line 1: `!<arch>'
+```
+
+The `!<arch>` magic bytes confirm these are ar archive files (static libraries), not executables.
+
+**Root Cause**: Unknown - possibly:
+- Git Bash / MSYS2 environment issue on Windows
+- Go toolchain configuration problem
+- Build script or Makefile issue specific to MinIO codebase
+- Path or environment variable corruption
+
+## Findings
+
+### SDK Implementation Status: ✅ CORRECT
+
+The Rust SDK's AWS SigV4 implementation is correct:
+- Service name: `s3tables` ✓
+- Region: `us-east-1` ✓
+- Canonical request format: AWS compliant ✓
+- Header canonicalization: Lowercase, sorted, proper format ✓
+- Content SHA256: Correctly calculated and included ✓
+- Authorization header: Proper AWS4-HMAC-SHA256 format ✓
+
+### What Still Needs Investigation
+
+1. **Server-Side Canonical Request**: Cannot compare without running modified server
+   - Need to see what the server calculates for the same request
+   - Check for differences in URI encoding (e.g., `%1F` for special characters)
+   - Verify header ordering and formatting matches
+
+2. **Credentials**: Verify `henk` user exists with correct credentials on server
+   ```bash
+   # Check with mc admin user list
+   mc admin user list myminio
+   ```
+
+3. **Region Configuration**: Ensure server's global site region is `us-east-1`
+   ```bash
+   # Check server config
+   mc admin config get myminio region
+   ```
+
+4. **Branch Status**: Confirm `tp/register-table` branch in C:\Source\minio\eos has complete Tables API implementation
+
+## Recommendations
+
+### Option 1: Build Server on Linux/Mac
+The MinIO build system is designed for Unix-like systems. Building on Linux or Mac should work correctly:
+```bash
+cd /path/to/eos
+make build
+./minio server /data
+```
+
+### Option 2: Use Pre-built Binary
+If a working MinIO binary with Tables API support is available, use that instead of building from source.
+
+### Option 3: Use WSL
+Build the server in Windows Subsystem for Linux:
+```bash
+wsl
+cd /mnt/c/Source/minio/eos
+make build
+```
+
+### Option 4: Docker
+Run MinIO in Docker with debug logging:
+```bash
+docker run -p 9000:9000 -p 9001:9001 \
+  -e MINIO_ROOT_USER=henk \
+  -e MINIO_ROOT_PASSWORD=$MINIO_ROOT_PASSWORD \
+  minio/minio:latest server /data --console-address ":9001"
+```
+
+## Test Command
+
+Once server is running with debug logging:
+```bash
+cd C:\Source\minio\minio-rs
+env SERVER_ENDPOINT="http://localhost:9000/" \
+    ENABLE_HTTPS="false" \
+    ACCESS_KEY="henk" \
+    SECRET_KEY="$MINIO_ROOT_PASSWORD" \
+    cargo test --test test_tables_create_delete warehouse_create -- --nocapture
+```
+
+This will show SDK's canonical request on stderr and (with modified server) the server's calculation for comparison.
+
+## Files Changed
+
+### SDK
+- `src/s3/signer.rs` - Added canonical request debug output
+
+### Server (Not Successfully Built)
+- `cmd/signature-v4.go` - Added debug logging (lines 19, 22, 382-415)
+- `cmd/auth-handler.go` - Added debug logging (lines 21, 26, 716-739)
+- `cmd/tables-api-handlers.go` - Added debug logging (line 21, 89-107)
+
+## Next Steps
+
+1. Get a working MinIO server binary (Linux build, WSL, Docker, or existing binary)
+2. Apply debug logging patches to server code
+3. Build and run server with debug output
+4. Run SDK test to capture both client and server canonical requests
+5. Compare the two canonical requests to identify any discrepancies
+6. Apply fix once specific difference is identified
+7. Remove all debug logging once issue is resolved