fix(honcho): restore cache-stable system prompt for turn-varying recall#1203
Closed
erosika wants to merge 2 commits intoNousResearch:mainfrom
Closed
fix(honcho): restore cache-stable system prompt for turn-varying recall#1203erosika wants to merge 2 commits intoNousResearch:mainfrom
erosika wants to merge 2 commits intoNousResearch:mainfrom
Conversation
…stability Commit 047b118 reintroduced per-turn Honcho context injection into the system prompt, reverting the cache-stability fix from aedb773. The system prefix must stay identical across turns for Anthropic/OpenRouter/OpenAI prompt caching to work. Move _honcho_turn_context into a separate <honcho-context> user message injected after system + prefill, keeping the system prefix stable while still delivering fresh recall to the model on continuing turns.
Contributor
Author
|
Closing in favor of #1201 which takes the better approach of appending recall to the current-turn user message rather than injecting a synthetic user message. Also handles multimodal content and includes regression tests. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
<honcho-context>message positioned after system + prefillDetail
When
_honcho_turn_contextis appended directly toeffective_system, the outbound system message changes on every turn. All three major caching mechanisms (Anthropic prompt caching, OpenRouter sticky routing, OpenAI prefix matching) depend on an exact-match system prefix to avoid cache invalidation.This change injects the turn-varying recall as a standalone user-role message instead, preserving cache-hit eligibility while still surfacing Honcho context to the model. First-turn behavior is unchanged (context is baked into the cached system prompt once at session start).
Test plan
<honcho-context>visible in debug logs)