Skip to content

Conversation

@yorickvP
Copy link
Contributor

@yorickvP yorickvP commented Nov 5, 2025

Problem

Running a profiler on my pinecone-using application. it was CPU-bottlenecked on json_format.MessageToDict (being able to do about 100 vectors per second in query and fetch responses).
It turns out converting the embeddings to a dict this way is very slow. It's much faster to convert them to a list without going through MessageToDict.

Solution

Changed parse_fetch_response and parse_query_response to directly read the protobuf structure instead of going through MessageToDict

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update
  • Infrastructure change (CI configs, etc)
  • Non-code change (docs, etc)
  • None of the above: (explain here)

Test Plan

Describe specific steps for validating this change.

  • Ran make test-grpc-unit.

Running a profiler on my pinecone program, it was CPU-bottlenecked
on json_format.MessageToDict (being able to do about 100 vectors per
second in query and fetch responses).
It turns out converting the embeddings to a dict this way is very
slow. It's much faster to convert them to a list without going through
MessageToDict.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant