-
Notifications
You must be signed in to change notification settings - Fork 10
Optimization: Reduce latency in LLM_CHAIN Execution #664
Copy link
Copy link
Open
Description
Is your feature request related to a problem?
The sequential execution of links for llm_chain causes latency due to dependencies on langfuse traces and webhook completion. This impacts user experience unnecessarily since these tasks are not central to it.
Describe the solution you'd like
- Optimize the execution flow to minimize latency.
- Allow tasks to proceed as soon as successful responses are received from llm_calls.
Original issue
Currently the sequential execution of links for llm_chain creates dependency on langfuse traces and webhook being completed. This spikes the latency while these two tasks are not central to the user experience perspective. So ideally, the tasks should move forward as soon as the response from the respective llm_calls are successful
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels
Type
Projects
Status
To Do