nvfp4: why do we need to call weight.contiguous for Qwen3 during lm-eval?

[vkuzo]

TODO @andrewor14 add repro

[andrewor14]

I can't seem to reproduce this anymore, my attempt:

import torch
from torchao.prototype.mx_formats import NVFP4InferenceConfig
from transformers import AutoModelForCausalLM, AutoTokenizer, TorchAoConfig

model_id = "Qwen/Qwen3-8B"
quantization_config = TorchAoConfig(quant_type=NVFP4InferenceConfig())
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    dtype=torch.bfloat16,
    quantization_config=quantization_config,
)
tokenizer = AutoTokenizer.from_pretrained(model_id)
save_to = "/tmp/qwen3-8b-nvfp4"
model.save_pretrained(save_to, safe_serialization=False)
tokenizer.save_pretrained(save_to)

Eval command (this works):

accelerate launch -m lm_eval --model hf --model_args pretrained="/tmp/qwen3-8b-nvfp4" --tasks mmlu --batch_size 32

My versions:

torchao commit: dffb3a0 (10/28/25)
torch: 2.8.0+cu128
lm_eval: 0.4.9.1
accelerate: 1.10.1

Maybe @SalmanMohammadi can share a repro if you still have one?