Skip to content

Streamable content only starts streaming after it's been completely streamed from the model #374

@heyitsaamir

Description

@heyitsaamir

The whole point of streaming is that the user starts receiving chunks before the whole message has been downloaded. Right now, if you put logs in, you can see that the chunks are fully downloaded, and THEN the stream starts to actually flow to Teams. This is wrong. We need to send chunks out as soon as they come in (barring rate limitations).

Hint:
The main issue is that the timeout gets reset any time a new chunk comes in.

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions