Engram docs

[g-despot]

What's being changed:

• Add Engram documentation: introduction, quickstart, concepts, guides (store, search, manage, run status), and API reference
• Integrate interactive API reference via Scalar + OpenAPI spec
• Add Engram to secondary navbar and sidebar

New pages

All under docs/engram/: introduction, quickstart, core concepts (memories, topics, groups, scoping, pipelines, runs), four how-to guides, and an API reference overview linking to the interactive Scalar reference at /engram/api-reference/rest.

Config changes

• sidebars.js — engramSidebar
• secondaryNavbar.js — Engram nav entry
• docusaurus.config.js — Scalar plugin instance for Engram OpenAPI spec
• static/specs/engram-openapi.json — Generated OpenAPI spec (user-facing endpoints only)

Type of change:

• Documentation content updates (non-breaking change to fix/update documentation )
• Bug fix (non-breaking change to fixes an issue with the site)
• Feature or enhancements (non-breaking change to add functionality)

How has this been tested?

• Local build - the site works as expected when running yarn start

[orca-security-eu]

Orca Security Scan Summary

Status	Check	Issues by priority
Passed	Infrastructure as Code	0   0   0   0	View in Orca
Passed	SAST	0   0   0   0	View in Orca
Passed	Secrets	0   0   0   0	View in Orca
Passed	Vulnerabilities	0   0   0   0	View in Orca

[augustas1]

we shouldn't pass base_url. that's the whole idea of SDK to encapsulate things like this.

[g-despot]

Good point, will remove this completely

[augustas1]

maybe we can put some static string to show that user_id can be simple user name? Field is called user_id for only reason to emphasize that it has to be unique.

[g-despot]

I'm using this because I ran into problems with deduplication with the same user id when running the tests multiple times, I will see if I can remove it

[augustas1]

gotcha. My point is, use some "readable" string instead of uuid, so users don't think we require uuid here.

[danmichaeljones]

The default topic is now called UserKnowledge with the description Anything relating to the user personally: their personal details (name, age, interpersonal relationships, etc.), their preferences, what they've done or plan to do, etc., i.e., any generic information about the user.

The template also lets you optionally add a ConversationSummary topic which maintains a single summary for each conversation ID which contains the entire history of that conversation so far. Enabling this option makes the conversation_id required when adding memories (which is why it's disabled by default).

[danmichaeljones]

Pipelines can also contain buffer steps. When a pipeline run hits one of these buffers, it pauses and waits for one of several possible triggers to flush and continue the pipeline run (it's during this pause that the run status is in_buffer).

These buffers don't have to be at the start of the pipeline (e.g., it's not just queueing and waiting to start running), they can be in the middle too.

One use case for this is aggregated memories over time. For example, we can have a pipeline which has steps like

[extract] -> [transform] -> [commit] -> [buffer] -> [transform] -> [commit]

where the buffer is configured to trigger after 24hrs. This pipeline would immediately extract/transform/commit memories as they're added. Then, those memories get sent to the buffer. That buffer accumulates all the memories that get added with the same scope (i.e., user/conversation ID) into a single batch that runs through the rest of the pipeline after it triggers. In this case, the second transform step could be configured to combine all those memories into a single "daily activity" memory which is committed by the second commit step.

The default personalisation pipeline doesn't include a buffer at the moment, so hopefully this will be a bit clearer once we have something using it (the continual learning template will have a buffer I think, and it'll be available as a step once pipelines are fully user-configurable).

[danmichaeljones]

Each triggers a different extraction pipeline

It's not quite a different pipeline for each content type, it's a different entrypoint into that pipeline (that's why the pipelines are DAGs). For example, a pipeline might look like

ExtractFromConversation        ExtractFromString
         │                              │       
         │                              │       
         └──────►TransformWithContext◄──┘       
                          │                     
                          ▼                     
                        Commit                  

so if you add conversation data, it'd pass through the steps

[ExtractFromConversation] -> [TransformWithContext] -> [Commit]

and if you added string data it'd pass through the steps

[ExtractFromString] -> [TransformWithContext] -> [Commit]

and that TransformWithContext is exactly the same step with the same config etc. That distinction is important when you include buffer steps, because if you route memories extracted from different content type into the same buffer step, they get aggregated together for later transformation steps.

[g-despot]

Thank you a lot for the detailed descriptions, they are amazing for the docs 💚

[danmichaeljones]

I don't know the best way to make it clear in the docs, but we want to separate the idea that adding conversation content type data means you need a conversation ID. They're separate concepts, so you can add conversation data (list of role/content) without a conversation_id, and conversely could include a conversation_id for string / pre-extracted content types.

Whether the conversation_id is required when adding or not just depends on the topic in that group, not the content type. I'd worry that only including that kwarg when adding conversation data in these examples conflates those two.

[danmichaeljones]

I don't think we want to encourage this generally. The idea is that most of the time people shouldn't be waiting on the pipelines completing, the memories should just be eventually consistent.

While the status could change to failed part way through a pipeline running, that should only be because of internal errors. It's probably best IMO to encourage the user to just check the run status that is returned from memories.add to see if it errored immediately.

[danmichaeljones]

Similar comment here as on the run status page - we don't need to wait after adding memories, and definitely not blocking the chat response.

It's useful to note that memories are most useful between conversation contexts or from very far back in the conversation etc. We generally don't need to wait until memories from the last message have been added to retrieve them for the next message, because the previous message will still be in context for the LLM.

[danmichaeljones]

This tutorial would probably be a good place to demonstrate the optional Include Conversation Summary Topic checkbox in the personalisation template, since that's basically built for exactly this use case.

Once that is enabled, it adds a ConversationSummary topic to the group (conversation-scoped, so all adds need a conversation ID when it's enabled). The pipeline keeps that topic updated with the history of the conversation so far, and always rewrites it with updates, i.e., every unique conversation_id has just one memory with the ConversationSummary topic.

Then in retrieval, you can just fetch that memory by using the fetch retrieval type (which I've just noticed is enabled in backend but not the python SDK so will TODO that!) using limit 1 and passing this conversation's conversation_id, and you should always get an up to date history of the entire conversation so far that you can use to replace old messages just like you did here.

[g-despot]

Will include this as soon as it's available in the Python SDK