Skip to content

Conversation

@shen-shanshan
Copy link
Collaborator

@shen-shanshan shen-shanshan commented Oct 30, 2025

What this PR does / why we need it?

Fix #3891.

The empty of moe_comm_method in the above issue is due to the wrong check for MoE models. To be specific, the method is_moe_model only checks whether a text-only model is a MoE model, without considering multi-modal models, e.g., VL and Omni.

Config of text-only models looks like:

We can verify MoE models by checking if there is a key named num_experts.

This is different from VL or Omni models, whose config files look like:

We can verify MoE models by checking if there is a key named num_experts in text_config dict.

Part of #3508.

Does this PR introduce any user-facing change?

How was this patch tested?


Update: 2025/11/03

Check the config dict recursively to find if it has a key contains "expert", without checking the model architecture.

It is worth noting that, we can't verify a model by if it contains FusedMoE module because is_moe_model is called somewhere before the model loading, e.g., it's called when updating the ACLGraph config in platform initialization.

@github-actions
Copy link

👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:‌‌

  • A PR should do only one thing, smaller PRs enable faster reviews.
  • Every PR should include unit tests and end-to-end tests ‌to ensure it works and is not broken by other future PRs.
  • Write the commit message by fulfilling the PR description to help reviewer and future developers understand.

If CI fails, you can run linting and testing checks locally according Contributing and Testing.

@shen-shanshan shen-shanshan changed the title [MM][Bugfix] Add MoE check for multi-modal models [MM][Bugfix] Add MoE verification for multi-modal models Oct 30, 2025
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request fixes a bug in is_moe_model to correctly identify multi-modal MoE models. The change introduces a check for text_config in multi-modal model configurations. While the overall logic is sound, I've identified a couple of areas for improvement. Firstly, a new global variable is modified within a function, which should be a module-level constant. Secondly, the method for detecting MoE models can be made more precise and efficient by directly checking for the num_experts key, as mentioned in the PR description, instead of using a substring search. My review comments provide specific suggestions to address these points.

@shen-shanshan
Copy link
Collaborator Author

Signed-off-by: shen-shanshan <467638484@qq.com>
Signed-off-by: shen-shanshan <467638484@qq.com>
@shen-shanshan
Copy link
Collaborator Author

CC @wangxiyuan

@shen-shanshan
Copy link
Collaborator Author

The CI has passed @wangxiyuan

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

module:core ready read for review ready-for-test start test by label for PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Run Qwen3-VL-30B-A3B-Instruct failed. moe_comm_method is None: 'NoneType' object has no attribute 'prepare'

3 participants