-
Notifications
You must be signed in to change notification settings - Fork 270
Add http_request and http_response hooks for LLM provider HTTP interception #345
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
Lovely, I planned on adding this, did an exploration, and also ran into the Google provider issue. I think it's fine to have it even if the Google provider doesn't support it. I'm finishing the session-tree refactor today, so this will have to be rebased. See #316 |
eabd123 to
4c1cc10
Compare
|
@badlogic This has been rebased, thx! |
9d5a8a7 to
3d76a21
Compare
|
@badlogic ready for |
|
Sorry, had to finish the session-tree branch. Will give it a looksy asap. |
|
@badlogic rebased |
|
@badlogic Please take a look before it conflicts again :) |
Adds HTTP-level hooks that allow intercepting requests to LLM providers for logging, adding custom headers, authentication injection, and request cancellation.
New Hook Events
http_requestFires before each HTTP request to an LLM provider. Handlers receive:
provider,modelId,url,methodheaders(sensitive headers redacted)body(string bodies only)Handlers can return:
{ headers: {...} }to add/override headers{ cancel: true }to abort the requesthttp_responseFires after each HTTP response. Handlers receive:
provider,modelId,status,headers,durationMsExample Hook
Implementation
fetchoption toStreamOptions, threaded through all provider functionscreateFetchfactory option toAgentOptionsandAgentLoopConfigHookRunnermethods, fetch wrapper factory in SDKProvider Support
The
@google/genaiSDK does not expose custom fetch in its public API. Users needing HTTP hooks with Google models should usegoogle-gemini-cliprovider.Security
Sensitive headers are automatically redacted in hook events:
authorization,x-api-key,api-key,x-goog-api-key,anthropic-api-key,proxy-authorization,cookie,set-cookieauth,token,key,secret, orcookieTests
Added 36 tests covering: