Feat(UI): Add LLM-powered prompt expansion and image-to-prompt features

[Pfannkuchensack]

Summary

Adds two new buttons to the positive prompt area:

• "Expand Prompt" uses a local TextLLM model (AutoModelForCausalLM) to expand brief prompts into detailed image generation prompts
• "Image to Prompt" uses an existing LLaVA OneVision model to generate descriptive prompts from uploaded images

Backend: New TextLLM model type with config, loader, pipeline wrapper, workflow node, and two new API endpoints (expand-prompt, image-to-prompt). Also fixes HuggingFace metadata fetch assertion error when file size is None.

Frontend: ExpandPromptButton and ImageToPromptButton components with model picker popovers, RTK Query mutations, and model type hooks. Buttons only appear when compatible models are installed.

Why

#8430

QA Instructions

1. Model install: Install a causal LM model (e.g. Qwen/Qwen2.5-1.5B-Instruct) via Model Manager — it should be recognized as text_llm type
2. Expand Prompt: Type a short prompt, click the sparkle button, select the TextLLM model, click "Expand" — the prompt should be replaced with an expanded version
3. Image to Prompt: Click the image button, select a LLaVA model, upload an image, click "Generate Prompt" — a descriptive prompt should appear
4. Conditional rendering: Buttons should only appear when compatible models are installed
5. HF metadata fix: Installing models from HuggingFace repos where some files have None size should no longer crash

Merge Plan

No special merge considerations. Standard merge to main.

Checklist

• The PR has a short but descriptive title, suitable for a changelog
• Tests added / updated (if applicable)
• ❗Changes to a redux slice have a corresponding migration
• Documentation added / updated (if applicable)
• Updated What's New copy (if doing a release after this PR)

[JPPhoto]

• Use model execution device for text LLM inference - /mnt/AI/InvokeAI3/src/invokeai/app/invocations/text_llm.py:56-61

When a text LLM is configured with cpu_only=True (or otherwise cached on CPU), model_on_device() keeps the model on CPU, but the pipeline inputs are moved to TorchDevice.choose_torch_device() which selects GPU when available. That device mismatch triggers runtime errors like "Expected all tensors to be on the same device" during generation. Consider deriving the device from the loaded model (e.g., next(model.parameters()).device) or the cache's execution device instead of a global device chooser.

• Utilities LLM endpoints ignore cpu_only device selection - /mnt/AI/InvokeAI3/src/invokeai/app/api/routers/utilities.py:93-99

Both _run_expand_prompt and _run_image_to_prompt move inputs to TorchDevice.choose_torch_device() regardless of the model's configured execution device. If the model is set to cpu_only=True (or the cache selects CPU), this causes GPU-bound inputs with a CPU model and breaks inference. Use the actual device of the loaded model or cache record instead of the global chooser to avoid device-mismatch failures when CPU-only models are used.

[JPPhoto]

@Pfannkuchensack A few more things that you might want to address:

1. Unbounded max_tokens in new API can cause excessive generation/OOM

• utilities.py:65 (invokeai/app/api/routers/utilities.py:65) defines max_tokens without bounds.
• text_llm invocation caps this at <=2048, but /v1/utilities/expand-prompt does not, so direct API calls can request arbitrarily large generations.
• Recommendation: make ExpandPromptRequest.max_tokens a pydantic Field, add limits (e.g. max_tokens: int = Field(default=300, ge=1, le=2048)).

2. Endpoint uses assert for request/model validation and maps all failures to HTTP 500 (typical for Invoke but we could do better!)

• asserts validate user-controlled model selection (utilities.py:155 (invokeai/app/api/routers/utilities.py:155), utilities.py:159 (invokeai/app/api/routers/utilities.py:159)),
• then a blanket catch maps those failures to 500 (utilities.py:125 (invokeai/app/api/routers/utilities.py:125), utilities.py:191 (invokeai/app/api/routers/utilities.py:191)).
• Invalid model_key/wrong model type should be a client error (4xx), not internal server error; asserts are also not robust validation behavior.
• Recommendation: replace asserts with explicit checks, log errors, and raise HTTPException(status_code=400/404/422, ...); keep 500 only for unexpected failures.

3. New Text LLM loader uses global dtype instead of execution-device-aware dtype

• text_llm.py:25 (invokeai/backend/model_manager/load/model_loaders/text_llm.py:25) loads with torch_dtype=self._torch_dtype.
• self._torch_dtype is chosen once from global preferred device, not from the model's eventual execution device (which may be CPU when cpu_only=True).
• This can load CPU-only models in an unsuitable dtype (commonly fp16 on CUDA hosts), causing inference failures unrelated to placement.
• Recommendation: choose dtype from the model's effective execution device (or force safe CPU dtype when cpu_only=True).

4. Testing

There are no new tests despite new functionality. Are there any that you think could be added?

[joshistoast]

You have merge conflicts 😞