Skip to content

Optimization: Reduce latency in LLM_CHAIN Execution #664

@Prajna1999

Description

@Prajna1999

Is your feature request related to a problem?
The sequential execution of links for llm_chain causes latency due to dependencies on langfuse traces and webhook completion. This impacts user experience unnecessarily since these tasks are not central to it.

Describe the solution you'd like

  • Optimize the execution flow to minimize latency.
  • Allow tasks to proceed as soon as successful responses are received from llm_calls.
Original issue

Currently the sequential execution of links for llm_chain creates dependency on langfuse traces and webhook being completed. This spikes the latency while these two tasks are not central to the user experience perspective. So ideally, the tasks should move forward as soon as the response from the respective llm_calls are successful

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

Status

To Do

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions