Skip to content

feat: Add Modal cloud GPU inference support#9

Merged
grider-transwithai merged 14 commits intoTransWithAI:mainfrom
Randomless:feature-modal
Jan 11, 2026
Merged

feat: Add Modal cloud GPU inference support#9
grider-transwithai merged 14 commits intoTransWithAI:mainfrom
Randomless:feature-modal

Conversation

@Randomless
Copy link
Copy Markdown
Contributor

@Randomless Randomless commented Dec 31, 2025

新增 modal_infer.py，用于通过 Modal 平台实现基于云端 GPU 的推理功能：

  • 提供交互式命令行界面（CLI），支持 GPU 选择与配置
  • 支持多种 GPU 类型（如 T4、A10G、A100、H100 等）
  • 自动在 Modal Volume 上下载并缓存模型
  • 支持批量处理，加速语音转录
  • 基于 Micromamba 构建的镜像，使用 environment-cuda128.yml 中定义的 conda 环境

新增 environment-modal.yml，用于轻量级本地客户端环境配置：

  • 仅包含最小依赖（modal、questionary）
  • 使用 Python 3.10 环境，以便在本地运行 modal_infer.py
  • 更新《使用说明.txt》，加入 Modal 平台使用指南：

环境配置说明

  • Modal 账号注册及 Token 配置方法
  • HuggingFace Token 设置（用于模型下载）
  • 分步操作指南

grider-transwithai and others added 13 commits December 26, 2025 14:27
@grider-transwithai @claude
fix: Pin PyInstaller to 6.16.0 to avoid conda hook regression
395d9b2 
PyInstaller 6.17.0 has a KeyError: 'depends' regression in some conda
environments. Pin to 6.16.0 until the issue is resolved.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@claude
feat: Add Modal cloud GPU inference support
fe20a3c 
Add modal_infer.py for cloud-based GPU inference using Modal platform:
- Interactive CLI for GPU selection and configuration
- Support for multiple GPU types (T4, A10G, A100, H100, etc.)
- Automatic model download and caching on Modal Volume
- Batch processing support for accelerated transcription
- Micromamba-based image with conda environment from environment-cuda128.yml

Add environment-modal.yml for lightweight local client setup:
- Minimal dependencies (modal, questionary)
- Python 3.10 environment for running modal_infer.py locally

Update 使用说明.txt with Modal usage instructions:
- Environment setup guide
- Modal account registration and token configuration
- HuggingFace token setup for model downloads
- Step-by-step usage instructions

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@claude
fix: Improve modal_infer.py usability and fix bugs
bf8d04b 
- Fix file upload path to use session directory consistently
- Simplify volume path structure (remove APP_ROOT_REL layer)
- Remove run mode selection, default to one-time execution
- Add detailed logging for build, execution and download stages
- Fix download conflict by adding --force flag
- Reorder GPU choices (T4/L4 first for cost efficiency)
- Update volume name to Faster_Whisper

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@Randomless @claude
refactor: 优化文件夹处理逻辑，逐文件上传避免连接断开
3f6ae4b 
- 文件夹模式下改为逐个上传和处理音频文件，容器复用
- 使用 min_containers=1 保持容器预热
- 排除 mp4 格式，发现时提示用户使用 ffmpeg 转换
- 文件夹模式下结果直接保存到源文件夹，不再创建 _out 子目录
- 单个文件处理失败时继续处理其他文件

注意：仅通过语法检查，未经手动测试验证

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@Randomless @claude
fix: 修复文件名包含空格导致远程执行失败的问题
a44f3ce 
- 上传时将文件名中的空格替换为下划线
- 在 UploadManifest 中记录原始文件名
- 下载字幕后恢复原始文件名（带空格）

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@Randomless @claude
fix: 使用固定文件名 todo 避免全角字符导致文件找不到
c3419a0 
- 上传时将文件名改为 todo + 扩展名，避免全角字符问题
- 下载后恢复原始文件名

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@Randomless @claude
debug: 推理失败时打印 sessions 目录和待处理文件目录内容
9bca526 
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@Randomless @claude
fix: 添加 Volume 同步等待，解决文件上传后容器看不到的问题
f4e81f0 
- 上传后调用 volume.commit() 强制同步
- 远程执行前等待文件出现（最多 30 秒）

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@Randomless @claude
fix: 移除本地 volume.commit() 调用（只能在容器内调用）
53ca0a2 
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@Randomless @claude
fix: 将文件同步等待时间延长至 2 分钟
9b6d464 
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@Randomless @claude
refactor: 优化日志和文件同步等待逻辑
304ddf5 
- log 文件放在 session 目录下，与 todo.mp3 同级
- 等待时间延长至 3 分钟
- 等待时打印 session 目录下的文件列表

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@Randomless @claude
fix: 在容器中调用 volume.reload() 强制同步文件
b50f5e6 
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@Randomless @claude
refactor: 通过函数返回值传输结果文件，避免 volume 同步问题
b15a784 
- 远程函数直接返回文件内容（base64 编码）
- 本地直接写入文件，使用原始文件名
- 移除 modal volume get 下载逻辑
- 删除不再使用的 RemoteResult 类

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@grider-transwithai
Copy link
Copy Markdown
Member

grider-transwithai commented Jan 10, 2026

请添加modal_infer.py到pyinstaller的spec里以生成相应exe，并测试CI是否通过，感谢

@Randomless
Copy link
Copy Markdown
Contributor Author

Randomless commented Jan 11, 2026

已添加pyinstaller spec

@grider-transwithai grider-transwithai merged commit 4d338c4 into TransWithAI:main Jan 11, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@Randomless @grider-transwithai