Skip to content

embedding api improvements #2249

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 26 commits into
base: master
Choose a base branch
from
Open

embedding api improvements #2249

wants to merge 26 commits into from

Conversation

@ricopinazo
Copy link
Collaborator

@ricopinazo ricopinazo commented Aug 28, 2025 •
edited
Loading

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change? If yes is this documented?

How was this patch tested?

Are there any further changes required?

  • make sure we actually saving the templates on disk in a test
  • maybe embeddings should not be enabled by default, which simply ignores any vectors on disk
  • make sure we have the appropriate blocking_io calls, especially around read/write operations on arroy
  • there might be some corner case where some old embedding sample is read from the cache but the actual model changed
  • rename scores to distances in rust/python/graphql apis
  • forbid graph names starting with a dot
  • make sure to add a graphql function to vectorise graphs on demand
  • remove arroy dependency
  • make sure we align our policy to ignore or not embedding errors so they are consistent across all APIs. Ideally, we would ignore them always, and simply retry embeddings later on so that the vectorised graph doesn't get out of sync.

@ricopinazo
embedding api improvements
f4cc018
@ricopinazo
disable embeddings by default
8bf8e8c
@ricopinazo
fix compilation error for enable_embeddings
6fa9962
@ricopinazo
fix compilation error in main.rs
3da0307
@ricopinazo
sort milvus integration
06640c0
@ricopinazo
lancedb vector storage implementation and rename score to distance
2b5b5f0
@ricopinazo
like this almost compiles but not yet
f906e94
@ricopinazo
force lancedb to work with other version of chrono but still polars-a…
976ff92 
…rrow complaining
@ricopinazo
this seems to compile with all features but storage
8e1975a
@ricopinazo
put pometry-storage back into place
e942d0e
@ricopinazo
sort new multi-embedding vector cache
5eeb703
@ricopinazo
still some rust tests failing
f637544
@ricopinazo
remove outdated comment
8b999da
@ricopinazo
all rust tests now passing with the new custom openai server
98f2555
@ricopinazo
some compilation errors caused by teh server future not being Sync
b1c66c7
@ricopinazo
fixing some python tests
9952530
@ricopinazo
wip
d7fe667
@ricopinazo
trying to avoid the drop of the tempdir but still not working
f0af2a9
@ricopinazo
fix compilation error
78e7701
@ricopinazo
fix bug caused by a temp dir being dropped too soon
3701786
@ricopinazo
fix python tests
705d588
@ricopinazo
Merge branch 'master' into embedding-api
ed89cea 
# Conflicts:
#	Cargo.lock
#	Cargo.toml
#	raphtory-graphql/schema.graphql
#	raphtory-graphql/src/python/server/server.rs
#	raphtory-graphql/src/server.rs
#	raphtory/Cargo.toml
@ricopinazo
fix dependency conflicts
f4ba9cc
@ricopinazo
format
857c5ab
github-actions[bot]
github-actions bot reviewed Oct 24, 2025
View reviewed changes
Copy link
Contributor

@github-actions github-actions bot left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Performance Alert ⚠️

Possible performance regression was detected for benchmark 'GraphQL Benchmark'.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 2.

Benchmark suite Current: 517c322 Previous: 675dfd2 Ratio
addNode 21 req/s 1314 req/s 62.57

This comment was automatically generated by workflow using github-action-benchmark.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@github-actions github-actions[bot] github-actions[bot] left review comments

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@ricopinazo