Introduce Megatron-style parallel state management#7726

Open

eternalNight wants to merge 18 commits intodeepspeedai:masterfrom

openanolis:eternalNight/unify_process_group_management

Commits on Jan 26, 2026

Introduce Megatron-style parallel state management

言枢
authored and
eternalNight
committed
parallel_state: Cleanup dependency on ProcessGroupNCCL.Options
eternalNight
committed
feat: add config-based parallel state initialization with validation

言枢
authored and
eternalNight
committed
Add sequence parallel support to refactored parallel state

yunqing
authored and
eternalNight
committed
fix: remove Chinese comment from config example

言枢
authored and
eternalNight
committed
fix: use torch.distributed.new_group directly in _create_group

言枢
authored and
eternalNight
committed
fix: correct SP parallel group creation logic in parallel_state

yunqing
authored and
eternalNight
committed
refactor: simplify _create_group to use deepspeed.comm interface

言枢
authored and
eternalNight
committed
feat: migrate All-to-All groups to parallel_state architecture

言枢
authored and
eternalNight
committed
fix: disable gloo process groups by default

言枢
authored and
eternalNight
committed
refactor: simplify SP group creation using RankGenerator

yunqing
authored and
eternalNight
committed
docs: fix config example and SP usage notes

言枢
authored and
eternalNight
committed
refactor: remove unused is_torch_min_version function

言枢
authored and
eternalNight
committed
refactor: simplify config-based initialization to use top-level fields

言枢
authored and
eternalNight
committed
tests: Drop test_mpu.py from the PR
eternalNight
committed

Commits on Feb 11, 2026

fix: filter unsupported params in initialize_parallel_state_from_config and add integration tests
yunqing
committed

Commits on Mar 2, 2026