Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 0 additions & 5 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -1,8 +1,3 @@
[submodule "3rdparty/Megatron-LM"]
path = 3rdparty/Megatron-LM-workspace/Megatron-LM
url = https://github.com/NVIDIA/Megatron-LM.git
branch = main
shallow = true
[submodule "3rdparty/Megatron-Bridge"]
path = 3rdparty/Megatron-Bridge-workspace/Megatron-Bridge
url = https://github.com/NVIDIA-NeMo/Megatron-Bridge.git
Expand Down
1 change: 0 additions & 1 deletion 3rdparty/Megatron-LM-workspace/Megatron-LM
Submodule Megatron-LM deleted from 23dd63
21 changes: 0 additions & 21 deletions 3rdparty/Megatron-LM-workspace/is_megatron_installed.py

This file was deleted.

12 changes: 0 additions & 12 deletions 3rdparty/Megatron-LM-workspace/pyproject.toml

This file was deleted.

194 changes: 0 additions & 194 deletions 3rdparty/Megatron-LM-workspace/setup.py

This file was deleted.

6 changes: 3 additions & 3 deletions docs/design-docs/dependency-management.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ A **development workflow** is when you actively modify dependencies, submodules,

- **Version mismatch**: Using a container built from commit A, but your local NeMo RL code is at commit B, where B has different submodule versions or Python dependencies than A
- **Dependency changes**: Actively developing new features that require updated Python packages
- **Submodule modifications**: Working with modified versions of Megatron-LM, NeMo-Automodel, or other submodules
- **Submodule modifications**: Working with modified versions of Megatron-Bridge, NeMo-Automodel, or other submodules

> [!WARNING]
> If your container was built from commit `abc123` which used `vllm==0.9.0`, but your local checkout is at commit `def456` which requires `vllm==0.10.0`, you are in a development workflow. The container's cached environments won't match your code's requirements.
Expand Down Expand Up @@ -161,7 +161,7 @@ The rebuilt container will have all virtual environments pre-cached with your up

### Option 3: Classic Workflow - Mounting Modified Submodules

For situations where you're **only changing submodules** (like nemo-automodel, NeMo Gym, Megatron-LM, or Megatron-Bridge) but **not changing Python package versions**, you can use a classic mounting approach. This workflow assumes that the non-submodule Python packages in your local checkout match what the container was built with.
For situations where you're **only changing submodules** (like nemo-automodel, NeMo Gym, or Megatron-Bridge, or Megatron-LM which is a submodule in Megatron-Bridge) but **not changing Python package versions**, you can use a classic mounting approach. This workflow assumes that the non-submodule Python packages in your local checkout match what the container was built with.

The container's NeMo RL code is located at `/opt/nemo-rl`. By mounting your local `3rdparty/` directory over the container's `/opt/nemo-rl/3rdparty/`, you can swap out submodules without rebuilding environments or containers.

Expand Down Expand Up @@ -193,7 +193,7 @@ This mounts:
> [!IMPORTANT]
> This workflow is **only suitable when**:
> - Python package versions in `pyproject.toml` and `uv.lock` haven't changed
> - You're only modifying code within submodules (nemo-automodel, NeMo Gym, Megatron-LM, Megatron-Bridge)
> - You're only modifying code within submodules (nemo-automodel, NeMo Gym, Megatron-LM within Megatron-Bridge, Megatron-Bridge)
> - The submodule commits/branches are compatible with the installed package versions

If you've changed Python package versions or dependencies outside of submodules, use Option 1 (`NRL_FORCE_REBUILD_VENVS=true`) or Option 2 (rebuild the container) instead.
Expand Down
4 changes: 2 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -167,7 +167,7 @@ test = [
]

[tool.uv.sources]
megatron-core = { workspace = true }
megatron-core = { path = "3rdparty/Megatron-Bridge-workspace/Megatron-Bridge/3rdparty/Megatron-LM", editable = true }
nemo-automodel = { workspace = true }
megatron-bridge = { workspace = true }
nemo_gym = { workspace = true }
Expand All @@ -192,7 +192,6 @@ sgl-kernel = { git = "https://github.com/sgl-project/sglang", rev = "3167bcc01c3

[tool.uv.workspace]
members = [
"3rdparty/Megatron-LM-workspace",
"3rdparty/Automodel-workspace/Automodel",
"3rdparty/Megatron-Bridge-workspace",
"3rdparty/Gym-workspace/Gym",
Expand Down Expand Up @@ -253,6 +252,7 @@ override-dependencies = [
"llguidance>=1.3.0,<1.4.0",
# Override setuptools range in other dependencies to address CVE GHSA-58pv-8j8x-9vj2
"setuptools>=80.10.2",
"nvidia-cutlass-dsl>=4.3.1",
]
# CVE fixes
constraint-dependencies = [
Expand Down
Loading
Loading