-
Notifications
You must be signed in to change notification settings - Fork 17
Engram docs #345
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
g-despot
wants to merge
24
commits into
main
Choose a base branch
from
engram
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+5,021
−348
Open
Engram docs #345
Changes from all commits
Commits
Show all changes
24 commits
Select commit
Hold shift + click to select a range
301bac7
Add initial docs
g-despot bc28bd0
Update docs
g-despot 1b934bb
Remove duplicate reference
g-despot a129d89
Merge branch 'main' into engram
g-despot 8157717
Update docs
g-despot bd35628
Update docs
g-despot cbde720
Add three tutorials
g-despot 6ba99c6
Merge branch 'main' into engram
g-despot d8469d5
Update concepts
g-despot 4c90794
Update docs
g-despot 11e3181
Minor update
g-despot 567503c
Improve Engram docs
g-despot 4e54443
Improve code verification
g-despot 4ed4952
Update docs
g-despot 165cda3
Improve code
g-despot 85f9cab
Update concepts
g-despot f398fad
Minor updates
g-despot 70a714d
Improve diagrams and linking
g-despot 0a69c5a
Add links
g-despot ff9a955
Improve diagram
g-despot 5f40a64
Improve concepts
g-despot fc28b39
Link to REST API
g-despot 4c6bac3
Update diagrams
g-despot c4f56a6
Implement feedback
g-despot File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| import os | ||
| import time | ||
| import uuid | ||
| from engram import EngramClient | ||
|
|
||
| client = EngramClient( | ||
| api_key=os.environ["ENGRAM_API_KEY"] | ||
| ) | ||
|
|
||
| test_user_id = f"test-{uuid.uuid4().hex[:8]}" | ||
|
|
||
| # Setup: store a memory to get a run_id | ||
| run = client.memories.add( | ||
| "The user prefers dark mode", | ||
| user_id=test_user_id, | ||
| group="default", | ||
| ) | ||
|
|
||
| # START PollRun | ||
| status = client.runs.wait(run.run_id) | ||
|
|
||
| print(status.run_id) | ||
| print(status.status) | ||
| print(status.committed_operations) | ||
| # END PollRun | ||
|
|
||
| assert status.status == "completed" | ||
| assert status.committed_operations is not None | ||
| assert len(status.memories_created) >= 1 | ||
|
|
||
| time.sleep(2) # Allow tenant indexing to complete | ||
|
|
||
| # Cleanup | ||
| _all = client.memories.search(query="dark mode", user_id=test_user_id, group="default") | ||
| for _m in _all: | ||
| client.memories.delete(_m.id, user_id=test_user_id, group="default") | ||
|
|
||
| client.close() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,67 @@ | ||
| #!/bin/bash | ||
| set -euo pipefail | ||
|
|
||
| BASE_URL="${ENGRAM_BASE_URL:-https://api.engram.weaviate.io}" | ||
| USER_ID="test-curl-$(uuidgen | tr '[:upper:]' '[:lower:]' | head -c 8)" | ||
|
|
||
| : <<'DOCSNIPPETS' | ||
| # START PollRun | ||
| curl https://api.engram.weaviate.io/v1/runs/{run-id} \ | ||
| -H "Authorization: Bearer $ENGRAM_API_KEY" | ||
| # END PollRun | ||
| DOCSNIPPETS | ||
|
|
||
| # --- Test execution below --- | ||
|
|
||
| # Create a memory to get a run_id | ||
| RUN_ID=$(curl -s -X POST "$BASE_URL/v1/memories" \ | ||
| -H "Authorization: Bearer $ENGRAM_API_KEY" \ | ||
| -H "Content-Type: application/json" \ | ||
| -d '{ | ||
| "content": { | ||
| "type": "string", | ||
| "content": "The user prefers dark mode and uses VS Code." | ||
| }, | ||
| "user_id": "'"$USER_ID"'", | ||
| "group": "default" | ||
| }' | jq -r '.run_id') | ||
|
|
||
| echo "Run ID: $RUN_ID" | ||
|
|
||
| # Poll until completed | ||
| for i in $(seq 1 30); do | ||
| STATUS=$(curl -s "$BASE_URL/v1/runs/$RUN_ID" \ | ||
| -H "Authorization: Bearer $ENGRAM_API_KEY" | jq -r '.status') | ||
| echo "Status: $STATUS" | ||
| [ "$STATUS" = "completed" ] && break | ||
| sleep 2 | ||
| done | ||
| [ "$STATUS" = "completed" ] || { echo "FAIL: run did not complete"; exit 1; } | ||
|
|
||
| # Verify committed_operations | ||
| CREATED_COUNT=$(curl -s "$BASE_URL/v1/runs/$RUN_ID" \ | ||
| -H "Authorization: Bearer $ENGRAM_API_KEY" | jq '.committed_operations.created | length') | ||
| [ "$CREATED_COUNT" -ge 1 ] || { echo "FAIL: expected at least 1 created memory"; exit 1; } | ||
| echo "Created memories: $CREATED_COUNT" | ||
|
|
||
| # Cleanup | ||
| for i in $(seq 1 10); do | ||
| WARMUP=$(curl -s -X POST "$BASE_URL/v1/memories/search" \ | ||
| -H "Authorization: Bearer $ENGRAM_API_KEY" \ | ||
| -H "Content-Type: application/json" \ | ||
| -d '{"query": "dark mode", "user_id": "'"$USER_ID"'", "group": "default", "retrieval_config": {"retrieval_type": "hybrid", "limit": 1}}') | ||
| WARMUP_COUNT=$(echo "$WARMUP" | jq '.memories | length' 2>/dev/null || echo "0") | ||
| [ "$WARMUP_COUNT" -ge 1 ] 2>/dev/null && break | ||
| sleep 3 | ||
| done | ||
|
|
||
| curl -s -X POST "$BASE_URL/v1/memories/search" \ | ||
| -H "Authorization: Bearer $ENGRAM_API_KEY" \ | ||
| -H "Content-Type: application/json" \ | ||
| -d '{"query": "user", "user_id": "'"$USER_ID"'", "group": "default", "retrieval_config": {"retrieval_type": "hybrid", "limit": 100}}' \ | ||
| | jq -r '.memories[]? | .id // empty' | while read -r MID; do | ||
| curl -s -X DELETE "$BASE_URL/v1/memories/$MID?user_id=$USER_ID&group=default" \ | ||
| -H "Authorization: Bearer $ENGRAM_API_KEY" > /dev/null | ||
| done | ||
|
|
||
| echo "PASS" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| import os | ||
| from engram import EngramClient | ||
|
|
||
| client = EngramClient( | ||
| api_key=os.environ["ENGRAM_API_KEY"] | ||
| ) | ||
|
|
||
| user_id = os.environ.get("ENGRAM_USER_ID", "user-uuid") | ||
| group = os.environ.get("ENGRAM_GROUP", "default") | ||
|
|
||
| deleted = 0 | ||
| for _ in range(20): | ||
| results = client.memories.search(query="user", user_id=user_id, group=group) | ||
| if len(results) == 0: | ||
| break | ||
| for m in results: | ||
| try: | ||
| client.memories.delete(m.id, topic=m.topic, user_id=user_id) | ||
| deleted += 1 | ||
| except Exception: | ||
| pass | ||
|
|
||
| print(f"Deleted {deleted} memories for user_id={user_id}, group={group}") | ||
|
|
||
| client.close() |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,58 @@ | ||
| import os | ||
| import uuid | ||
| from engram import EngramClient, APIError | ||
|
|
||
| client = EngramClient( | ||
| api_key=os.environ["ENGRAM_API_KEY"] | ||
| ) | ||
|
|
||
| test_user_id = f"test-{uuid.uuid4().hex[:8]}" | ||
|
|
||
| # Setup: store a memory so we can get and delete it | ||
| run = client.memories.add( | ||
| "The user prefers dark mode", | ||
| user_id=test_user_id, | ||
| group="default", | ||
| ) | ||
| status = client.runs.wait(run.run_id) | ||
| assert status.status == "completed" | ||
| assert status.committed_operations is not None | ||
| assert len(status.memories_created) >= 1 | ||
|
|
||
| memory_id = status.memories_created[0].memory_id | ||
|
|
||
| # START GetMemory | ||
| memory = client.memories.get( | ||
| memory_id, | ||
| user_id=test_user_id, | ||
| group="default", | ||
| ) | ||
|
|
||
| print(memory.content) | ||
| print(memory.topic) | ||
| # END GetMemory | ||
|
|
||
| assert memory.id == memory_id | ||
| assert "dark mode" in memory.content | ||
|
|
||
| # START DeleteMemory | ||
| client.memories.delete( | ||
| memory_id, | ||
| user_id=test_user_id, | ||
| group="default", | ||
| ) | ||
| # END DeleteMemory | ||
|
|
||
| # Verify the memory was deleted | ||
| try: | ||
| client.memories.get(memory_id, user_id=test_user_id, group="default") | ||
| assert False, "Expected memory to be deleted" | ||
| except APIError: | ||
| pass # Memory no longer exists | ||
|
|
||
| # Cleanup | ||
| _all = client.memories.search(query="dark mode", user_id=test_user_id, group="default") | ||
| for _m in _all: | ||
| client.memories.delete(_m.id, user_id=test_user_id, group="default") | ||
|
|
||
| client.close() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,90 @@ | ||
| #!/bin/bash | ||
| set -euo pipefail | ||
|
|
||
| BASE_URL="${ENGRAM_BASE_URL:-https://api.engram.weaviate.io}" | ||
| USER_ID="test-curl-$(uuidgen | tr '[:upper:]' '[:lower:]' | head -c 8)" | ||
|
|
||
| : <<'DOCSNIPPETS' | ||
| # START GetMemory | ||
| curl https://api.engram.weaviate.io/v1/memories/{memory-id}?user_id=user-uuid&group=default \ | ||
| -H "Authorization: Bearer $ENGRAM_API_KEY" | ||
| # END GetMemory | ||
|
|
||
| # START DeleteMemory | ||
| curl -X DELETE https://api.engram.weaviate.io/v1/memories/{memory-id}?user_id=user-uuid&group=default \ | ||
| -H "Authorization: Bearer $ENGRAM_API_KEY" | ||
| # END DeleteMemory | ||
| DOCSNIPPETS | ||
|
|
||
| # --- Test execution below --- | ||
|
|
||
| # Seed a memory to manage | ||
| RUN_ID=$(curl -s -X POST "$BASE_URL/v1/memories" \ | ||
| -H "Authorization: Bearer $ENGRAM_API_KEY" \ | ||
| -H "Content-Type: application/json" \ | ||
| -d '{ | ||
| "content": { | ||
| "type": "string", | ||
| "content": "The user prefers dark mode and uses VS Code." | ||
| }, | ||
| "user_id": "'"$USER_ID"'", | ||
| "group": "default" | ||
| }' | jq -r '.run_id') | ||
|
|
||
| for i in $(seq 1 30); do | ||
| STATUS=$(curl -s "$BASE_URL/v1/runs/$RUN_ID" \ | ||
| -H "Authorization: Bearer $ENGRAM_API_KEY" | jq -r '.status') | ||
| [ "$STATUS" = "completed" ] && break | ||
| sleep 2 | ||
| done | ||
| [ "$STATUS" = "completed" ] || { echo "FAIL: seed run did not complete"; exit 1; } | ||
|
|
||
| # Wait for indexing | ||
| for i in $(seq 1 10); do | ||
| WARMUP=$(curl -s -X POST "$BASE_URL/v1/memories/search" \ | ||
| -H "Authorization: Bearer $ENGRAM_API_KEY" \ | ||
| -H "Content-Type: application/json" \ | ||
| -d '{"query": "dark mode VS Code", "user_id": "'"$USER_ID"'", "group": "default", "retrieval_config": {"retrieval_type": "hybrid", "limit": 1}}') | ||
| WARMUP_COUNT=$(echo "$WARMUP" | jq '.memories | length' 2>/dev/null || echo "0") | ||
| [ "$WARMUP_COUNT" -ge 1 ] 2>/dev/null && break | ||
| sleep 3 | ||
| done | ||
|
|
||
| # Find a memory ID | ||
| MEMORY_ID=$(curl -s -X POST "$BASE_URL/v1/memories/search" \ | ||
| -H "Authorization: Bearer $ENGRAM_API_KEY" \ | ||
| -H "Content-Type: application/json" \ | ||
| -d '{"query": "dark mode", "user_id": "'"$USER_ID"'", "group": "default", "retrieval_config": {"retrieval_type": "hybrid", "limit": 5}}' \ | ||
| | jq -r '.memories[0].id') | ||
|
|
||
| [ "$MEMORY_ID" != "null" ] || { echo "FAIL: no memory found to manage"; exit 1; } | ||
| echo "Memory ID: $MEMORY_ID" | ||
|
|
||
| # Test get memory | ||
| GOT_ID=$(curl -s "$BASE_URL/v1/memories/$MEMORY_ID?user_id=$USER_ID&group=default" \ | ||
| -H "Authorization: Bearer $ENGRAM_API_KEY" | jq -r '.id') | ||
| [ "$GOT_ID" = "$MEMORY_ID" ] || { echo "FAIL: get returned wrong memory"; exit 1; } | ||
| echo "Get memory: OK" | ||
|
|
||
| # Test delete memory | ||
| curl -s -X DELETE "$BASE_URL/v1/memories/$MEMORY_ID?user_id=$USER_ID&group=default" \ | ||
| -H "Authorization: Bearer $ENGRAM_API_KEY" | ||
| echo "Delete memory: OK" | ||
|
|
||
| # Verify deletion | ||
| HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" \ | ||
| "$BASE_URL/v1/memories/$MEMORY_ID?user_id=$USER_ID&group=default" \ | ||
| -H "Authorization: Bearer $ENGRAM_API_KEY") | ||
| [ "$HTTP_CODE" = "404" ] || echo "Warning: expected 404 after delete, got $HTTP_CODE" | ||
|
|
||
| # Cleanup remaining | ||
| curl -s -X POST "$BASE_URL/v1/memories/search" \ | ||
| -H "Authorization: Bearer $ENGRAM_API_KEY" \ | ||
| -H "Content-Type: application/json" \ | ||
| -d '{"query": "user", "user_id": "'"$USER_ID"'", "group": "default", "retrieval_config": {"retrieval_type": "hybrid", "limit": 100}}' \ | ||
| | jq -r '.memories[]? | .id // empty' | while read -r MID; do | ||
| curl -s -X DELETE "$BASE_URL/v1/memories/$MID?user_id=$USER_ID&group=default" \ | ||
| -H "Authorization: Bearer $ENGRAM_API_KEY" > /dev/null | ||
| done | ||
|
|
||
| echo "PASS" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,66 @@ | ||
| import os | ||
| import time | ||
| import uuid | ||
| from engram import EngramClient | ||
|
|
||
| # START Connect | ||
| client = EngramClient( | ||
| api_key=os.environ["ENGRAM_API_KEY"] | ||
| ) | ||
| # END Connect | ||
|
|
||
| test_user_id = f"test-{uuid.uuid4().hex[:8]}" | ||
|
|
||
| # START AddMemory | ||
| run = client.memories.add( | ||
| "The user prefers dark mode and uses VS Code as their primary editor.", | ||
| user_id=test_user_id, | ||
| ) | ||
|
|
||
| print(run.run_id) | ||
| print(run.status) | ||
| # END AddMemory | ||
|
|
||
| assert run.run_id is not None | ||
|
|
||
| # START CheckRun | ||
| status = client.runs.wait(run.run_id) | ||
|
|
||
| print(status.status) | ||
| # END CheckRun | ||
|
|
||
| assert status.status == "completed" | ||
| assert status.committed_operations is not None | ||
| assert len(status.memories_created) >= 1 | ||
|
|
||
| from engram.errors import APIError | ||
|
|
||
| # Warm up tenant — retry until search succeeds (tenant may still be initializing) | ||
| for _retry in range(5): | ||
| try: | ||
| client.memories.search(query="test", user_id=test_user_id, group="default") | ||
| break | ||
| except APIError: | ||
| time.sleep(3) | ||
|
|
||
| # START SearchMemory | ||
| results = client.memories.search( | ||
| query="What editor does the user prefer?", | ||
| user_id=test_user_id, | ||
| ) | ||
|
|
||
| for memory in results: | ||
| print(memory.content) | ||
| # END SearchMemory | ||
|
|
||
| assert len(results) >= 1 | ||
| assert any("VS Code" in m.content or "editor" in m.content or "dark mode" in m.content for m in results) | ||
|
|
||
| # Cleanup | ||
| for _m in results: | ||
| try: | ||
| client.memories.delete(_m.id, user_id=test_user_id, group="default") | ||
| except Exception: | ||
| pass | ||
|
|
||
| client.close() | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
maybe we can put some static string to show that
user_idcan be simple user name? Field is calleduser_idfor only reason to emphasize that it has to be unique.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm using this because I ran into problems with deduplication with the same user id when running the tests multiple times, I will see if I can remove it
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
gotcha. My point is, use some "readable" string instead of uuid, so users don't think we require uuid here.