Old veScale Has been Moved to legacy/; New veScale is Coming ...

veScale

veScale is an internal PyTorch Distributed library, enabling hyperscale distributed training of LLMs and RLs. This repo open-sources a small piece of veScale for a better community.

Content

• Quick Start

• RaggedShard DTensor

Publication

• veScale 2025-2026: Coming Soon

• veScale 2023-2025: https://arxiv.org/abs/2509.07003

Citation

@misc{li2025vescale,
      title={veScale: Consistent and Efficient Tensor Programming with Eager-Mode SPMD}, 
      author={Youjie Li and Cheng Wan and Zhiqi Lin and Hongyu Zhu and Jiacheng Yang and Ziang Song and Xinyi Di and Jiawei Wu and Huiyao Shu and Wenlei Bao and Yanghua Peng and Haibin Lin and Li-Wen Chang},
      year={2025},
      eprint={2509.07003},
      archivePrefix={arXiv},
      primaryClass={cs.PL},
      url={https://arxiv.org/abs/2509.07003}, 
}

License

The veScale Project is under the Apache License v2.0.