O1 model doesn'ts support streaming completions

[sheldonhull]

Background

With the introduction of newer reasoning models in v1.7.0, we've seen that while some tweaks have been made, streaming support for models like o1 was initially unavailable. This support varies based on the provider:

• OpenAI: Recently added streaming support.
• Azure OpenAI: Still lacks streaming support for o1.

Problem

The current implementation assumes that all results are streamed, which is incompatible with some o1 models. They require complete chat completion before results can be read, creating a gap in configuration and logic, especially for Azure OpenAI users.

Proposal

1. Add a new configuration value:
   • stream: false (default to true).
2. Introduce a CLI flag for manual override:
   • Use --stream false or --no-stream.

Seem reasonable? If so I'll see if I can knock that out.

ref: current limitations of o1 in azure openai

[gwpl]

AI Assistant:

We can confirm this issue affects o3/o3-mini/o3-pro models as well, not just o1.

Error Details

When attempting to use o3 models without organization verification for streaming:

$ echo test | mods -m o3
   ERROR  openai API request error.  

  POST "https://api.openai.com/v1/chat/completions": 400 Bad Request {
      "message": "Your organization must be verified to stream this model. Please go to: https://platform.openai.com/settings/organization/general and click on Verify Organization. If you just verified, it can take up to 15 minutes for access to propagate.",
      "type": "invalid_request_error",
      "param": "stream",
      "code": "unsupported_value"
    }

The --raw flag does not help - the error persists because streaming is hardcoded in internal/openai/openai.go:97.

Impact

This affects all reasoning models (o1, o3 series) where:

• Organizations aren't verified for streaming
• Azure OpenAI deployments don't support streaming
• API providers have streaming restrictions

Request

The proposed solution in the original issue (adding stream: false config option per model + --no-stream CLI flag) would resolve this for all affected models.

Thanks for your work on mods!

[gwpl]

AI Assistant:

Draft PR Available: #619

I've created a draft PR implementing the --no-stream flag as proposed in this issue: #619

Testing Results

The fix has been tested and confirmed working with o3 models:

WITHOUT --no-stream (fails):

$ echo "test" | ./mods -m o3
ERROR: 400 Bad Request
"message": "Your organization must be verified to stream this model..."
"param": "stream"
"code": "unsupported_value"

WITH --no-stream (works!):

$ echo "test" | ./mods -m o3 --no-stream
Hello! It looks like you're testing the chat. How can I help you today?
Conversation saved: 6f469bf test

Implementation Scope

This PR provides a minimal fix by adding only the CLI flag. Per-model configuration in config files is intentionally left for future work to keep the change focused and minimal.

The implementation:

• Adds --no-stream CLI flag
• Uses Chat Completion API (non-streaming) when flag is set
• Maintains full backward compatibility (streaming still default)
• Includes NonStreamingWrapper for consistent interface

All tests pass, and both streaming and non-streaming modes work correctly.