Problem with WebGPU (windows with amd gpu)

[DarkSorrow]

System Info

transfomer.js 3.8.1
brave navigator
windows system AMD gpu with react 19.2
gpu on
Adrenalin Edition 26.1.1

Environment/Platform

• Website/web-app
• Browser extension
• Server-side (e.g., Node.js, Deno, Bun)
• Desktop app (e.g., Electron)
• Other (e.g., VSCode extension)

Description

Hello,
Most of the time i have no issues but sometime i'm passing some tool results to the LLM loaded (here i'm with qwen3 4b) and i'm getting this type of error.

I'll paste the error down i think its mostly a driver problem i feel rather than tarnsformers itself but i'm guessing people hererr would have more insight than me :p

Reproduction

ai.js?v=ccd72f33:8695 Failed to execute 'createBuffer' on 'GPUDevice': Failed to read the 'size' property from 'GPUBufferDescriptor': Value is outside the 'unsigned long long' value range.
onError	@	ai.js?v=ccd72f33:8695
transform	@	ai.js?v=ccd72f33:8896
ID3D12Device::GetDeviceRemovedReason failed with DXGI_ERROR_DEVICE_HUNG (0x887A0006)
 - While handling unexpected error type Internal when allowed errors are (Validation|DeviceLost).
    at CheckHRESULTImpl (..\..\third_party\dawn\src\dawn\native\d3d\D3DError.cpp:121)

Backend messages:
 * Device removed reason: DXGI_ERROR_DEVICE_HUNG (0x887A0006)
app:1 ID3D12Device::GetDeviceRemovedReason failed with DXGI_ERROR_DEVICE_HUNG (0x887A0006)
 - While handling unexpected error type Internal when allowed errors are (Validation|DeviceLost).
    at CheckHRESULTImpl (..\..\third_party\dawn\src\dawn\native\d3d\D3DError.cpp:121)

Backend messages:
 * Device removed reason: DXGI_ERROR_DEVICE_HUNG (0x887A0006)


This happens with large tool result (despite the context not being above 8k as i control it).

[xenova]

Hi @DarkSorrow - thanks for the report! How much VRAM do you have? My assumption is that the browser decided you were using too much of your available memory and reset the connection.

[DarkSorrow]

I have 12gb on the card, its true that i'm sending a big chunk when i get that error and i think its probably what you said but the thing i don't get is that i get alert message before telling me "unable to determine buffer size" and such. Also i was passing dtype alongside the options for the webgpu but i get that message:
dtype not specified for "model". Using the default dtype (fp32) for this device (webgpu).

I think i need to debug a bit better, somehow there is a bit of a difference from what i get in production and in local dev. I made a custom webworker with react and i'm not sure if its acting a bit differently

[xenova]

Could you provide a sample of your code?

import { pipeline, TextStreamer } from "@huggingface/transformers";

// Create a text generation pipeline
const generator = await pipeline(
  "text-generation",
  "onnx-community/Qwen3-0.6B-ONNX",
  { device: "webgpu", dtype: "q4f16" },
);

// Define the list of messages
const messages = [
  { role: "system", content: "You are a helpful assistant." },
  { role: "user", content: "Write me a poem about Machine Learning." },
];

// Generate a response
const output = await generator(messages, {
  max_new_tokens: 512,
  do_sample: false,
  streamer: new TextStreamer(generator.tokenizer, { skip_prompt: true, skip_special_tokens: true }),
});
console.log(output[0].generated_text.at(-1).content);

It should look something like this, just replacing with your model ID

[DarkSorrow]

i'll try to extract part of it. Its still work in progress with some testing

My model is loaded fom db like this

'mdu': 'onnx-community/Qwen3-0.6B-ONNX',
'task': 'text-generation',
'options': '{"dtype": "q4f16"}',

But i'll have to make sure the dtype is properly passed because on prod i get a warning that its not present

  const token_callback_function = (tokens: any) => {
     if (tokens && tokens.length > 0 && currentTokenizer.eos_token_id !== undefined) {
       // remove that shit later
       tokens.some((t: any) => Number(t) === Number(currentTokenizer.eos_token_id));
     }
   };

   const streamer = new TextStreamer(currentTokenizer, {
     skip_prompt: true,
     skip_special_tokens: true,
     callback_function: (text: string) => {
       postMessage({
         type: 'TOKEN_STREAM',
         payload: {
           messageId,
           token: text,
         },
       });
     },
     token_callback_function,
   });

   let inputs;
   try {
     inputs = currentTokenizer.apply_chat_template(messages, {
       add_generation_prompt: true,
       return_dict: true,
     });
   } catch (e) {
     const lastMessage = messages[messages.length - 1];
     let simplePrompt = lastMessage?.content || '';

     if (state.config.systemPrompt && messages.length === 1) {
       simplePrompt = `${state.config.systemPrompt}\n\n${simplePrompt}`;
     }

     inputs = currentTokenizer(simplePrompt, {
       return_dict: true,
       add_special_tokens: true,
     });
   }

   const generationOptions = {
     ...inputs,
     do_sample: true,
     top_k: Number.isInteger(state.config.generation?.topK) ? state.config.generation?.topK : 20,
     temperature: typeof state.config.generation?.temperature === 'number' ? state.config.generation?.temperature : 0.7,
     max_new_tokens: typeof state.config.contextLength === 'number' ? state.config.contextLength : 24528,
     streamer: streamer,
     stopping_criteria: stoppingCriteria,
     return_dict_in_generate: true,
   };

   let sequences;
   try {
     sequences = (await currentGenerator.generate(generationOptions)).sequences;
   } catch (error: any) {
     // Any error during generation - send ERROR message so provider can reject
     const errorMessage = error?.message || String(error);
     const errorCode = error?.code || error?.statusCode || undefined;

     postMessage({
       type: 'ERROR',
       payload: {
         messageId,
         message: errorMessage,
         error: errorMessage,
         code: errorCode,
       },
     });
     return;
   }

   if (interruptedMessages.has(messageId)) {
     interruptedMessages.delete(messageId);
     return;
   }

   const generatedText = currentTokenizer.batch_decode(sequences, {
     skip_special_tokens: true,
   })[0];