Add support for Deferred Functions

[heyitsaamir]

Deferred Function Calling

(Found out later that Pydantic AI also has a feature like this - https://ai.pydantic.dev/deferred-tools/)

Motivation

Normally, when a function call happens in an AI system, it looks like this:

sequenceDiagram
    participant CP as Chat Prompt
    participant LLM
    participant Fn as Function
    CP ->> LLM: Messages, system prompt<br/>function metadatas
    LLM ->> CP: Call Fn1 with arguments X
    CP ->> Fn: execute(X)
    Fn ->> CP: result (string)
    CP ->> LLM: Messages + Fn Result

Loading

This works for many use-cases. In this example, we're assuming F1 is some function that can be called and resolved in a reasonable amount of time (maybe a few seconds).

However, there are some scenarios where the function cannot be easily (or quickly) be executed. Examples include:

• Requires human input or approval - Humans are slow. You might have gone to get coffee and your approval is sticking around forever. Here's an example from Claude Code:

• The function needs to execute in Business Hours, or next month, or next year!
• The result of the function call can take time to execute. Consider having to send an order to a system that can take a day to process.

Proposal

In this PR, I'm proposing the ability for functions to return a DeferredResult in addition to a string. The idea is that if the ChatPrompt sees "DeferredResult", it saves that state, and stops executing.
Then, when the system is ready (aka gets an external signal) to continue, then it "resumes" and continues the ChatPrompt. The updated sequence diagram for a Deferrable Function looks like so:

sequenceDiagram
    participant Teams
    participant CP as Chat Prompt
    participant LLM
    participant Fn as Deferrable Function
    Teams ->> CP: send("buy some stock") (via Message handler)
    CP ->> LLM: Messages, system prompt<br/>function metadatas
    LLM ->> CP: Call Fn1 with arguments X
    CP ->> Fn: execute(X)
    Fn ->> Teams: Ask for approval
    Fn ->> CP: DeferredResult(state)
    CP ->> Teams: No response
    break When signal is received
        Teams ->> CP: "yes" go ahead<br/>resume(input)
        CP ->> Fn: Can this resume using input?
        Fn ->> CP: true
        CP ->> Fn: Resume(input)
        Fn ->> CP: Result after resuming (string)<br/>"the user approved the buying of the stock"
    end
    CP ->> LLM: Messages + Fn Result

Loading

Implementation

• We add a new type of Message that can be stored in Memory (which can store Message which is a union of different types of messages). This DeferredMessage basically stores the deferred state that the executable function can optionally return.
• A function can return DeferredResult now (on top of string)
• Added a "resume" function in ChatPrompt that can take in "Any" right now (Ideally I want to make this "Activity", but not totally sure). This resume function checks with memory if a what the deferred function is, then asks the deferred function if it can handle this incoming activity, and if it can, it converts this DeferredMessage with a FunctionMessage which is a normal function call result.
• Introduce an "ApprovalPlugin" which makes use of this. It basically wraps the functions provided to it, and converts them into deferred functions that require approval.

Questions

1. Should we move the plugin to its own sample or utility? Might be better as a plugin that folks can modify
2. I don't really like the "should_resume" and "resume" methods. I wish I could just "send" to the ChatPrompt. But the send function takes in a Message not Activity. Thoughts?