External models (Gemini Nano Banana & OpenAI GPT Image) (#8633)

docs/contributing/EXTERNAL_PROVIDERS.md

@@ -0,0 +1,129 @@
+# External Provider Integration
+
+This guide covers:
+
+1. Adding a new **external model** (most common; existing provider).
+2. Adding a brand-new **external provider** (adapter + config + UI wiring).
+
+## 1) Add a New External Model (Existing Provider)
+
+For provider-backed models (for example, OpenAI or Gemini), the source of truth is
+`invokeai/backend/model_manager/starter_models.py`.
+
+### Required model fields
+
+Define a `StarterModel` with:
+
+- `base=BaseModelType.External`
+- `type=ModelType.ExternalImageGenerator`
+- `format=ModelFormat.ExternalApi`
+- `source="external://<provider_id>/<provider_model_id>"`
+- `name`, `description`
+- `capabilities=ExternalModelCapabilities(...)`
+- optional `default_settings=ExternalApiModelDefaultSettings(...)`
+
+Example:
+
+```python
+new_external_model = StarterModel(
+    name="Provider Model Name",
+    base=BaseModelType.External,
+    source="external://openai/my-model-id",
+    description=(
+        "Provider model (external API). "
+        "Requires a configured OpenAI API key and may incur provider usage costs."
+    ),
+    type=ModelType.ExternalImageGenerator,
+    format=ModelFormat.ExternalApi,
+    capabilities=ExternalModelCapabilities(
+        modes=["txt2img", "img2img", "inpaint"],
+        supports_negative_prompt=False,
+        supports_seed=False,
+        supports_guidance=False,
+        supports_steps=False,
+        supports_reference_images=True,
+        max_images_per_request=4,
+    ),
+    default_settings=ExternalApiModelDefaultSettings(
+        width=1024,
+        height=1024,
+        num_images=1,
+    ),
+)
+```
+
+Then append it to `STARTER_MODELS`.
+
+### Required description text
+
+External starter model descriptions must clearly state:
+
+- an API key is required
+- usage may incur provider-side costs
+
+### Capabilities must be accurate
+
+These flags directly control UI visibility and request payload fields:
+
+- `supports_negative_prompt`
+- `supports_seed`
+- `supports_guidance`
+- `supports_steps`
+- `supports_reference_images`
+
+`supports_steps` is especially important: if `False`, steps are hidden for that model and `steps` is sent as `null`.
+
+### Source string stability
+
+Starter overrides are matched by `source` (`external://provider/model-id`). Keep this stable:
+
+- runtime capability/default overrides depend on it
+- installation detection in starter-model APIs depends on it
+
+`STARTER_MODELS` enforces unique `source` values with an assertion.
+
+### Install behavior notes
+
+- External starter models are managed in **External Providers** setup (not the regular Starter Models tab).
+- External starter models auto-install when a provider is configured.
+- Removing a provider API key removes installed external models for that provider.
+
+## 2) Credentials and Config
+
+External provider API keys are stored separately from `invokeai.yaml`:
+
+- default file: `~/invokeai/api_keys.yaml`
+- resolved path: `<INVOKEAI_ROOT>/api_keys.yaml`
+
+Non-secret provider settings (for example base URL overrides) stay in `invokeai.yaml`.
+
+Environment variables are still supported, e.g.:
+
+- `INVOKEAI_EXTERNAL_GEMINI_API_KEY`
+- `INVOKEAI_EXTERNAL_OPENAI_API_KEY`
+
+## 3) Add a New Provider (Only If Needed)
+
+If your model uses a provider that is not already integrated:
+
+1. Add config fields in `invokeai/app/services/config/config_default.py`
+   `external_<provider>_api_key` and optional `external_<provider>_base_url`.
+2. Add provider field mapping in `invokeai/app/api/routers/app_info.py`
+   (`EXTERNAL_PROVIDER_FIELDS`).
+3. Implement provider adapter in `invokeai/app/services/external_generation/providers/`
+   by subclassing `ExternalProvider`.
+4. Register the provider in `invokeai/app/api/dependencies.py` when building
+   `ExternalGenerationService`.
+5. Add starter model entries using `source="external://<provider>/<model-id>"`.
+6. Optional UI ordering tweak:
+   `invokeai/frontend/web/src/features/modelManagerV2/subpanels/AddModelPanel/ExternalProviders/ExternalProvidersForm.tsx`
+   (`PROVIDER_SORT_ORDER`).
+
+## 4) Optional Manual Installation
+
+You can also install external models directly via:
+
+`POST /api/v2/models/install?source=external://<provider_id>/<provider_model_id>`
+
+If omitted, `path`, `source`, and `hash` are auto-populated for external model configs.
+Set capabilities conservatively; the external generation service enforces capability checks at runtime.

docs/contributing/index.md

@@ -8,6 +8,10 @@ We welcome contributions, whether features, bug fixes, code cleanup, testing, co
 
 If you’d like to help with development, please see our [development guide](contribution_guides/development.md).
 
+## External Providers
+
+If you are adding external image generation providers or configs, see our [external provider integration guide](EXTERNAL_PROVIDERS.md).
+
 **New Contributors:** If you’re unfamiliar with contributing to open source projects, take a look at our [new contributor guide](contribution_guides/newContributorChecklist.md).
 
 ## Nodes

invokeai/app/api/dependencies.py

@@ -16,6 +16,8 @@
 from invokeai.app.services.config.config_default import InvokeAIAppConfig
 from invokeai.app.services.download.download_default import DownloadQueueService
 from invokeai.app.services.events.events_fastapievents import FastAPIEventService
+from invokeai.app.services.external_generation.external_generation_default import ExternalGenerationService
+from invokeai.app.services.external_generation.providers import GeminiProvider, OpenAIProvider
 from invokeai.app.services.image_files.image_files_disk import DiskImageFileStorage
 from invokeai.app.services.image_records.image_records_sqlite import SqliteImageRecordStorage
 from invokeai.app.services.images.images_default import ImageService
@@ -145,13 +147,22 @@ def initialize(
             ),
         )
         download_queue_service = DownloadQueueService(app_config=configuration, event_bus=events)
-        model_images_service = ModelImageFileStorageDisk(model_images_folder / "model_images")
+        model_record_service = ModelRecordServiceSQL(db=db, logger=logger)
         model_manager = ModelManagerService.build_model_manager(
             app_config=configuration,
-            model_record_service=ModelRecordServiceSQL(db=db, logger=logger),
+            model_record_service=model_record_service,
             download_queue=download_queue_service,
             events=events,
         )
+        external_generation = ExternalGenerationService(
+            providers={
+                GeminiProvider.provider_id: GeminiProvider(app_config=configuration, logger=logger),
+                OpenAIProvider.provider_id: OpenAIProvider(app_config=configuration, logger=logger),
+            },
+            logger=logger,
+            record_store=model_record_service,
+        )
+        model_images_service = ModelImageFileStorageDisk(model_images_folder / "model_images")
         model_relationships = ModelRelationshipsService()
         model_relationship_records = SqliteModelRelationshipRecordStorage(db=db)
         names = SimpleNameService()
@@ -184,6 +195,7 @@ def initialize(
             model_relationships=model_relationships,
             model_relationship_records=model_relationship_records,
             download_queue=download_queue_service,
+            external_generation=external_generation,
             names=names,
             performance_statistics=performance_statistics,
             session_processor=session_processor,