Skip to content

Commit 52d1ed4

Browse files
NguyenSiTrungNguyenSiTrung
committed
conductor(s3-browse_20260310): create track for AWS S3 browse mode
Add read-only S3 browsing feature track with spec, plan, and learnings. - CLI activation: fm s3://bucket/path --aws-profile mfa - Backend: shell out to aws CLI (no SDK dependency) - Browse-only: list, navigate, on-demand preview - 5-phase plan: backend → tree → preview → UX → integration - Synced to Beads epic FileManagerTUI-qeb
1 parent 3766d85 commit 52d1ed4

5 files changed

Lines changed: 305 additions & 0 deletions

File tree

‎conductor/tracks.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -61,3 +61,8 @@
6161
---
6262

6363
<!-- Archived: preview-dblclick_20260304 (2026-03-05) → conductor/archive/ -->
64+
65+
---
66+
67+
## [ ] Track: AWS S3 Browse Mode (Read-Only via CLI)
68+
*Link: [./conductor/tracks/s3-browse_20260310/](./conductor/tracks/s3-browse_20260310/)*

‎conductor/tracks/s3-browse_20260310/learnings.md‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
# Track Learnings: s3-browse_20260310
2+
3+
Patterns, gotchas, and context discovered during implementation.
4+
5+
## Codebase Patterns (Inherited)
6+
7+
Key patterns from `conductor/patterns.md` relevant to this track:
8+
9+
- TreeState owns root TreeNode + flat_items Vec; `flatten()` rebuilds flat list from tree recursively
10+
- App delegates tree operations to TreeState methods; handler.rs maps keys to App methods
11+
- Handler uses 3-level dispatch: global keys → panel-specific keys → dialog keys
12+
- Store `event_tx: Option<mpsc::UnboundedSender<Event>>` on App to enable async ops from non-handler contexts
13+
- Keep sync fallback when `event_tx` is None for test contexts
14+
- Every `load_children()` call MUST be followed by `sort_children_of()`
15+
- Adding new `NodeType` variants requires updating ALL exhaustive matches: tree widget, ui.rs status bar, handler.rs
16+
- Graceful degradation for optional subsystems: wrap initialization in match, set state flag to false, show status message on error
17+
- Async operations via `tokio::spawn` + `mpsc::unbounded_channel` events integrate with the existing event loop
18+
- Use `Arc<AtomicBool>` for cancel tokens
19+
- OSC 52 clipboard fallback for headless/web terminal environments
20+
21+
---
22+
23+
<!-- Learnings from implementation will be appended below -->

‎conductor/tracks/s3-browse_20260310/metadata.json‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
{
2+
"track_id": "s3-browse_20260310",
3+
"type": "feature",
4+
"status": "new",
5+
"priority": "high",
6+
"depends_on": [],
7+
"estimated_hours": null,
8+
"created_at": "2026-03-10T19:37:10+07:00",
9+
"updated_at": "2026-03-10T19:37:10+07:00",
10+
"description": "Add read-only AWS S3 browsing mode via CLI shelling (aws s3 ls/cp) with on-demand download preview",
11+
"beads_epic": "FileManagerTUI-qeb",
12+
"beads_tasks": {
13+
"phase1": "FileManagerTUI-qeb.1",
14+
"phase2": "FileManagerTUI-qeb.2",
15+
"phase3": "FileManagerTUI-qeb.3",
16+
"phase4": "FileManagerTUI-qeb.4",
17+
"phase5": "FileManagerTUI-qeb.5"
18+
}
19+
}

‎conductor/tracks/s3-browse_20260310/plan.md‎

Lines changed: 151 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,151 @@
1+
# Plan: AWS S3 Browse Mode
2+
3+
## Phase 1: S3 Backend Module
4+
<!-- execution: sequential -->
5+
6+
- [ ] Task 1: Create S3 types and path parsing (`src/s3/mod.rs`, `src/s3/types.rs`)
7+
- [ ] Define `S3Path` struct (bucket, prefix/key)
8+
- [ ] Parse `s3://bucket/prefix/key` URIs into `S3Path`
9+
- [ ] Define `S3Entry` struct (name, is_dir, size, modified)
10+
- [ ] Define `S3Backend` struct (profile: Option<String>)
11+
- [ ] Unit tests for URI parsing (edge cases: root bucket, trailing slash, no prefix)
12+
13+
- [ ] Task 2: Implement `aws s3 ls` output parser (`src/s3/parser.rs`)
14+
- [ ] Parse `PRE <name>/` lines as directory entries
15+
- [ ] Parse `<date> <time> <size> <name>` lines as file entries
16+
- [ ] Handle empty output (empty prefix)
17+
- [ ] Handle error output from stderr
18+
- [ ] Unit tests with sample `aws s3 ls` output (dirs, files, mixed, empty, errors)
19+
20+
- [ ] Task 3: Implement async S3 listing via CLI (`src/s3/backend.rs`)
21+
- [ ] `S3Backend::list_prefix()` — spawn `aws [--profile] s3 ls s3://...` via `tokio::process::Command`
22+
- [ ] Capture stdout → parse with parser; capture stderr → error handling
23+
- [ ] `S3Backend::check_cli()` — verify `aws` exists on PATH
24+
- [ ] `S3Backend::download_to_cache()` — `aws s3 cp` to temp cache dir
25+
- [ ] Manage temp cache directory lifecycle (`/tmp/fm-s3-cache-<pid>/`)
26+
- [ ] Unit tests for command construction (verify args are correct)
27+
28+
- [ ] Task 4: Add CLI flags for S3 mode (`src/config.rs`, `src/main.rs`)
29+
- [ ] Add `--aws-profile <name>` flag to clap args
30+
- [ ] Detect `s3://` prefix in PATH argument
31+
- [ ] Store S3 config in `AppConfig` (s3_mode: bool, aws_profile: Option<String>, s3_path: Option<S3Path>)
32+
- [ ] Validate `aws` CLI at startup; exit with actionable error if missing
33+
- [ ] Tests for CLI arg parsing
34+
35+
- [ ] Task: Conductor - User Manual Verification 'S3 Backend Module' (Protocol in workflow.md)
36+
37+
## Phase 2: S3 Tree Integration
38+
<!-- execution: sequential -->
39+
40+
- [ ] Task 1: Add backend mode to App state (`src/app.rs`)
41+
- [ ] Add `BackendMode` enum (Local, S3 { backend: S3Backend })
42+
- [ ] Store `backend_mode` on `App` struct
43+
- [ ] Initialize `BackendMode::S3` when config has s3_path
44+
- [ ] Add `App::is_s3_mode()` helper
45+
- [ ] Disable filesystem watcher when in S3 mode
46+
47+
- [ ] Task 2: Build S3 TreeNodes from listing results (`src/fs/tree.rs`)
48+
- [ ] Add `TreeNode::from_s3_entry()` constructor — creates node without `fs::metadata`
49+
- [ ] Use S3 URI string as the `path` field (stored as PathBuf for compatibility)
50+
- [ ] Set `FileMeta` from `S3Entry` (size, modified, is_hidden)
51+
- [ ] S3 root node: `TreeNode::new_s3_root(s3_path)` — creates expandable root
52+
- [ ] Tests for S3 TreeNode construction
53+
54+
- [ ] Task 3: S3 directory expansion (`src/app.rs`, `src/fs/tree.rs`)
55+
- [ ] Override `expand_selected()` path for S3 mode — call `S3Backend::list_prefix()` instead of `fs::read_dir`
56+
- [ ] Build children from `Vec<S3Entry>` via `TreeNode::from_s3_entry()`
57+
- [ ] Async expansion with loading indicator (reuse `is_loading` + `DirScanComplete` event pattern)
58+
- [ ] Wire `DirScanComplete` handler to accept S3 listing results
59+
- [ ] Tests for S3 tree expansion flow
60+
61+
- [ ] Task 4: Disable write operations in S3 mode (`src/handler.rs`)
62+
- [ ] Guard `a`/`A` (create), `r` (rename), `d` (delete), `x` (cut), `p` (paste) keys
63+
- [ ] Show "Not available in S3 mode" status message when attempted
64+
- [ ] Disable `Ctrl+Z` undo in S3 mode
65+
- [ ] Disable inline editor (`e` key) in S3 mode
66+
- [ ] Disable `T` (open terminal at path) for S3 entries
67+
68+
- [ ] Task: Conductor - User Manual Verification 'S3 Tree Integration' (Protocol in workflow.md)
69+
70+
## Phase 3: S3 Preview & Clipboard
71+
<!-- execution: sequential -->
72+
73+
- [ ] Task 1: S3 metadata preview (`src/app.rs`, `src/preview_content.rs`)
74+
- [ ] When in S3 mode + selected item is file: render metadata preview (size, date, URI, download prompt)
75+
- [ ] Add `load_s3_metadata_preview()` function in `preview_content.rs`
76+
- [ ] Generate styled `Vec<Line<'static>>` showing S3 object info
77+
- [ ] For S3 directories: show prefix info + child count from listing
78+
79+
- [ ] Task 2: On-demand download and preview (`src/app.rs`, `src/handler.rs`)
80+
- [ ] Enter key on S3 file → spawn async download via `S3Backend::download_to_cache()`
81+
- [ ] Show loading indicator in preview panel during download
82+
- [ ] On download complete: pipe cached file through existing `update_preview()` pipeline
83+
- [ ] Cache tracking: `HashMap<String, PathBuf>` mapping S3 keys → local cache paths
84+
- [ ] Skip re-download if already cached for this session
85+
86+
- [ ] Task 3: S3 URI clipboard copy (`src/app.rs`, `src/handler.rs`)
87+
- [ ] `y` key on S3 file → copy `s3://bucket/key` string to clipboard
88+
- [ ] Use existing system clipboard + OSC 52 fallback path
89+
- [ ] Show "📋 Copied: s3://..." status message
90+
91+
- [ ] Task: Conductor - User Manual Verification 'S3 Preview & Clipboard' (Protocol in workflow.md)
92+
93+
## Phase 4: UX Polish & Error Handling
94+
<!-- execution: parallel -->
95+
96+
- [ ] Task 1: S3 status bar badge (`src/components/status_bar.rs`)
97+
<!-- files: src/components/status_bar.rs -->
98+
- [ ] Show `☁ S3 | s3://bucket` on the left section when in S3 mode
99+
- [ ] Replace local path display with S3 URI for selected item
100+
101+
- [ ] Task 2: S3 tree colors (`src/theme.rs`, `src/components/tree.rs`)
102+
<!-- files: src/theme.rs, src/components/tree.rs -->
103+
- [ ] Add `tree_s3_fg` color to `ThemeColors` (default: amber/orange #fab387)
104+
- [ ] Apply S3-specific color to S3 entries in tree widget rendering
105+
- [ ] Add S3 icon prefix: `☁` for S3 directories, `📦` for S3 objects
106+
107+
- [ ] Task 3: Loading indicators (`src/components/preview.rs`)
108+
<!-- files: src/components/preview.rs -->
109+
- [ ] Show `⏳ Downloading...` in preview panel during S3 file download
110+
- [ ] Show `⏳ Loading...` in tree during S3 directory expansion (reuse existing pattern)
111+
112+
- [ ] Task 4: Help overlay updates (`src/components/help.rs`)
113+
<!-- files: src/components/help.rs -->
114+
- [ ] Grey out disabled S3 keybindings with `DarkGray` color
115+
- [ ] Add "(S3: disabled)" suffix to write operation entries
116+
- [ ] Add S3-specific hints: "Enter: Download & preview", "y: Copy S3 URI"
117+
118+
- [ ] Task 5: Error handling polish (`src/s3/backend.rs`)
119+
<!-- files: src/s3/backend.rs -->
120+
- [ ] Parse common AWS CLI errors (ExpiredToken, AccessDenied, NoSuchBucket) into user-friendly messages
121+
- [ ] Show error dialog for authentication failures
122+
- [ ] Add retry prompt on network/timeout errors
123+
- [ ] Clean up temp cache directory on app exit
124+
125+
- [ ] Task: Conductor - User Manual Verification 'UX Polish & Error Handling' (Protocol in workflow.md)
126+
127+
## Phase 5: Integration & Cleanup
128+
<!-- execution: sequential -->
129+
<!-- depends: phase1, phase2, phase3, phase4 -->
130+
131+
- [ ] Task 1: Disable incompatible features (`src/app.rs`, `src/main.rs`)
132+
- [ ] Fuzzy search (Ctrl+P): show "Not available in S3 mode"
133+
- [ ] File watcher: skip initialization entirely
134+
- [ ] Terminal panel: allow toggle but `T` action on S3 entries shows message
135+
- [ ] Inline filter (`/`): works on currently loaded tree nodes (no S3 calls)
136+
137+
- [ ] Task 2: Temp cache cleanup and session management
138+
- [ ] Create unique cache dir per session: `/tmp/fm-s3-cache-<pid>/`
139+
- [ ] Register cleanup on app shutdown (in `main.rs` restore flow)
140+
- [ ] Handle Ctrl+C cleanup via existing panic hook
141+
142+
- [ ] Task 3: End-to-end manual testing checklist
143+
- [ ] Verify: `fm s3://bucket/prefix/ --aws-profile mfa` launches correctly
144+
- [ ] Verify: expanding S3 directories lists contents
145+
- [ ] Verify: file metadata preview shows without download
146+
- [ ] Verify: Enter downloads and shows syntax-highlighted preview
147+
- [ ] Verify: disabled operations show correct messages
148+
- [ ] Verify: status bar, colors, loading indicators work
149+
- [ ] Verify: error cases (missing CLI, bad credentials, no internet)
150+
151+
- [ ] Task: Conductor - User Manual Verification 'Integration & Cleanup' (Protocol in workflow.md)

‎conductor/tracks/s3-browse_20260310/spec.md‎

Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
# Spec: AWS S3 Browse Mode
2+
3+
## Overview
4+
5+
Add a read-only S3 browsing mode to `fm-tui` that lets users navigate and preview
6+
S3 bucket contents directly from the TUI, using the AWS CLI (`aws`) as the backend.
7+
This targets Kubeflow environments where AWS credentials and the CLI are already
8+
configured.
9+
10+
**Activation:** Passing an `s3://` path as the starting argument:
11+
```
12+
fm s3://my-bucket/experiments/ --aws-profile mfa
13+
```
14+
15+
**Backend:** Shell out to `aws s3 ls` / `aws s3 cp` commands — no AWS SDK
16+
dependency; leverages existing CLI authentication (profiles, IRSA, env vars).
17+
18+
## Functional Requirements
19+
20+
### FR-1: CLI Argument Parsing
21+
- Detect when the PATH argument starts with `s3://`
22+
- Accept `--aws-profile <name>` CLI flag (optional; omit for default profile)
23+
- Parse the S3 URI into bucket name and prefix components
24+
- Validate that the `aws` CLI is available on `$PATH`; show clear error if missing
25+
26+
### FR-2: S3 Tree Navigation
27+
- List S3 prefixes ("directories") and objects ("files") by running
28+
`aws [--profile <p>] s3 ls s3://bucket/prefix/`
29+
- Parse the CLI output to extract:
30+
- `PRE <name>/` → directory node
31+
- `<date> <time> <size> <name>` → file node with size and modified date
32+
- Build `TreeNode` entries from parsed output (virtual — no local path)
33+
- Support expand/collapse of S3 prefix directories (lazy-load on expand)
34+
- Support pagination for prefixes with many objects
35+
- Disable write operations in S3 mode: create (`a`/`A`), rename (`r`),
36+
delete (`d`), cut (`x`), paste (`p`) — show "Not available in S3 mode" message
37+
38+
### FR-3: S3 File Preview (On-Demand)
39+
- When navigating to an S3 object, show metadata preview (no download needed):
40+
- File size (human-readable)
41+
- Last modified date/time
42+
- Full S3 URI (key path)
43+
- Prompt: "Press [Enter] to download and preview"
44+
- On explicit Enter/`p` keypress:
45+
- Download to temp cache: `aws [--profile <p>] s3 cp s3://... /tmp/fm-s3-cache/...`
46+
- Show full syntax-highlighted preview using existing preview pipeline
47+
- Cache downloaded files for the session (avoid re-download on revisit)
48+
- `y` key copies the full `s3://` URI to clipboard (system or OSC 52 fallback)
49+
50+
### FR-4: S3-Specific UX Indicators
51+
- **Status bar badge:** Show `☁ S3 | s3://bucket` in status bar when in S3 mode
52+
- **Distinct tree colors:** Use a configurable S3-specific color (default: orange/
53+
amber) for S3 entry names to visually distinguish from local files
54+
- **Loading indicators:** Show `⏳ Loading...` during `aws` CLI calls (listing,
55+
downloading) since they have noticeable latency (~100-500ms per call)
56+
- **Disabled features:** Grey out inapplicable keybindings in the help overlay
57+
and status bar hints; show "Not available in S3 mode" toast on attempt
58+
59+
### FR-5: Error Handling
60+
- Handle `aws` CLI not found: show actionable error at startup
61+
- Handle authentication failures: display stderr from `aws` CLI as error dialog
62+
- Handle network/timeout errors: show error message, allow retry
63+
- Handle empty prefixes: show "Empty directory" in preview (same as local)
64+
65+
## Non-Functional Requirements
66+
67+
### NFR-1: No New Binary Dependencies
68+
- Shell out to `aws` CLI — no `aws-sdk-s3` crate
69+
- No increase in binary size beyond the new Rust code itself
70+
71+
### NFR-2: Latency Tolerance
72+
- All `aws` CLI calls must be async (spawned via `tokio::process::Command`)
73+
- UI must remain responsive during S3 operations (loading indicators)
74+
- Tree listing should not block the event loop
75+
76+
### NFR-3: Configuration
77+
- `--aws-profile` CLI flag
78+
- Future: TOML config for default profile, cache directory, timeout settings
79+
80+
### NFR-4: Graceful Degradation
81+
- Features that don't apply in S3 mode (watcher, inline editor, terminal `cd`)
82+
should be silently disabled — not crash
83+
- Fuzzy search disabled in S3 mode (would require full prefix enumeration)
84+
85+
## Acceptance Criteria
86+
87+
1. `fm s3://bucket/prefix/ --aws-profile mfa` opens the TUI showing S3 contents
88+
2. Expanding an S3 "directory" lists its children via `aws s3 ls`
89+
3. Selecting an S3 file shows metadata (size, date, URI) without downloading
90+
4. Pressing Enter on an S3 file downloads it and shows syntax-highlighted preview
91+
5. `y` on an S3 file copies the `s3://` URI to clipboard
92+
6. Write operations (create, rename, delete, cut, paste) show disabled message
93+
7. Status bar shows `☁ S3` badge; tree entries use distinct color
94+
8. Loading indicators appear during network operations
95+
9. Help overlay reflects S3-mode keybindings (write ops greyed out)
96+
10. Error messages are clear when `aws` CLI is missing or auth fails
97+
98+
## Out of Scope
99+
100+
- Write operations (create, rename, delete, move) on S3
101+
- AWS SDK integration (using CLI shelling only)
102+
- In-app switching between local and S3 modes
103+
- Dual-pane local+S3 view
104+
- S3 event notifications / live watching
105+
- Cross-region or cross-account browsing
106+
- File editing on S3 objects
107+
- Fuzzy search across S3 contents

0 commit comments

Comments
 (0)