NotImplementedError: Cannot copy out of meta tensor; no data! Please use torch.nn.Module.to_empty() instead of torch.nn.Module.to() when moving module from meta to a different device. #15
Description
saved_name = f"lora_model_{model_name.replace('/','_')}"
model.save_pretrained(saved_name)
tokenizer.save_pretrained(saved_name)
from unsloth import FastLanguageModel
model, tokenizer = FastLanguageModel.from_pretrained(
"lora_model_unsloth_llama-3-8b-bnb-4bit",
load_in_4bit = True,
max_seq_length = 1024,
dtype = None,
)
print("Model loaded successfully.")
==((====))== Unsloth 2025.8.1: Fast Llama patching. Transformers: 4.54.0.
\ /| NVIDIA A100-SXM4-40GB. Num GPUs = 1. Max memory: 39.557 GB. Platform: Linux.
O^O/ _/ \ Torch: 2.6.0+cu124. CUDA: 8.0. CUDA Toolkit: 12.4. Triton: 3.2.0
\ / Bfloat16 = TRUE. FA [Xformers = 0.0.29.post3. FA2 = False]
"-____-" Free license: http://github.com/unslothai/unsloth
Unsloth: Fast downloading is enabled - ignore downloading bars which are red colored!
NotImplementedError Traceback (most recent call last)
/tmp/ipython-input-1944140996.py in <cell line: 0>()
1 from unsloth import FastLanguageModel
----> 2 model, tokenizer = FastLanguageModel.from_pretrained(
3 "lora_model_epoch2_unsloth_llama-3-8b-bnb-4bit",
4 load_in_4bit = True,
5 max_seq_length = 1024,
11 frames
/usr/local/lib/python3.11/dist-packages/torch/nn/modules/module.py in convert(t)
1334 except NotImplementedError as e:
1335 if str(e) == "Cannot copy out of meta tensor; no data!":
-> 1336 raise NotImplementedError(
1337 f"{e} Please use torch.nn.Module.to_empty() instead of torch.nn.Module.to() "
1338 f"when moving module from meta to a different device."
NotImplementedError: Cannot copy out of meta tensor; no data! Please use torch.nn.Module.to_empty() instead of torch.nn.Module.to() when moving module from meta to a different device.