-
-
Notifications
You must be signed in to change notification settings - Fork 36.2k
Add chat log subscription endpoint #155287
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Hey there @home-assistant/core, mind taking a look at this pull request as it has been labeled with an integration ( Code owner commandsCode owners of
|
|
Hey there @home-assistant/core, @synesthesiam, @arturpragacz, mind taking a look at this pull request as it has been labeled with an integration ( Code owner commandsCode owners of
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR adds WebSocket subscription endpoints for chat log monitoring to improve debugging of LLM interactions in voice assistant sessions. The implementation allows real-time observation of conversation data exchanged between LLMs and Home Assistant.
Key changes:
- Added two WebSocket subscription endpoints for chat log access: individual conversation and index (all conversations)
- Implemented event notification system for chat log lifecycle (created, updated, deleted, content_added)
- Added
as_dict()serialization methods and timestamp tracking to content models
Reviewed Changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
homeassistant/components/conversation/http.py |
Implements WebSocket subscription handlers for chat logs |
homeassistant/components/conversation/chat_log.py |
Adds event subscription infrastructure and serialization methods |
homeassistant/components/conversation/const.py |
Defines chat log event type enumeration |
homeassistant/components/homeassistant/const.py |
Fixes typo in exposed entities key name |
tests/components/conversation/test_http.py |
Tests WebSocket subscription endpoints and event flows |
tests/components/conversation/test_chat_log.py |
Tests chat log subscription and event notification system |
|
I'm not a fan of the fact that this adds yet another mechanism to report events related to voice. |
|
@arturpragacz I hear you, however, chat log is not voice specific. It's the way we track the history of an interaction with an AI from either conversation or AI task. If we were to add these events into the Assist Pipeline websocket endpoint, we would now need to also make that one aware of AI task. Would you have another suggestion? |
Can we not have a way to register a callback in the chat log, which can then be used in the higher layer. So the assist pipeline would use it to provide those events through its websocket and the AI task could do what is the best for it. I don't think we have any debug interface for AI tasks at this point anyway, unless I'm missing something. |
|
This is the debugging endpoint for AI tasks. |
arturpragacz
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please address the open comments by Copilot.
🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
758d5ca to
e0ebc78
Compare
arturpragacz
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As discussed, let's merge it now, but investigate how to include conversation trace in this.
Breaking change
Proposed change
To improve debugging of LLMs, adding a set of endpoints to allow voice assistant debug sessions to look inside the chat log data to see what was sent between LLM and HA.
Frontend: home-assistant/frontend#27678
Type of change
Additional information
Checklist
ruff format homeassistant tests)If user exposed functionality or configuration variables are added/changed:
If the code communicates with devices, web services, or third-party tools:
Updated and included derived files by running:
python3 -m script.hassfest.requirements_all.txt.Updated by running
python3 -m script.gen_requirements_all.To help with the load of incoming pull requests: