Merged
Conversation
Add mission to align Rust CLI/library exports with Python SDK signatures: - completion/acompletion functions - embedding/aembedding functions - Router struct with routing strategies - LiteLLM-compatible exceptions - OpenAI-compatible proxy endpoints - LiteLLM-style CLI commands
Add detailed implementation design for RFC-0908 Python SDK: - Updated RFC with separate crate structure (quota-router-pyo3, quota-router-core) - Updated Mission 0908-a dependencies - Created design document with implementation steps
Prepare for isolated worktree development.
Create quota-router-core crate (extracted from CLI): - Move balance.rs, providers.rs, config.rs, proxy.rs - CLI now depends on core library Create quota-router-pyo3 crate: - PyO3 bindings for Python SDK - LiteLLM-compatible exceptions (AuthenticationError, RateLimitError, etc.) - Types: Message, ChatCompletion, Choice, Usage, Embedding - completion() and embedding() functions - Drop-in replacement for LiteLLM Build and test verified: - cargo build --all - cargo clippy --all-targets -- -D warnings - cargo test --all
Tried adding async support but pyo3-asyncio 0.20 has API differences: - async fn not directly supported as pyfunction - try_init_runtime not available in this version Keeping sync versions for now. Async (acompletion, aembedding) can be added when: - Python 3.11+ is minimum - pyo3-asyncio has stable async fn support - Or use manual Future polling approach All tests pass.
- Add pyproject.toml for maturin build system - Add .venv/ and .python-version to .gitignore - Verified with Python 3.12.9 Python SDK now pip-installable and working: - import quota_router works - completion() returns LiteLLM-compatible response - embedding() works - Exception classes available
Using pyo3 0.21 with experimental-async feature: - Add acompletion() async function - Add aembedding() async function - Verified working with Python 3.12 All sync and async functions working: - completion() - acompletion() - embedding() - aembedding()
- Add python/ directory with quota_router package - Add type stubs (__init__.pyi) for IDE support - Update root pyproject.toml for maturin - Add python/pyproject.toml for pip installability Python package now pip-installable: - import quota_router - import quota_router as litellm (LiteLLM alias) - completion(), acompletion(), embedding(), aembedding() - Exception classes
- docs/quota-router-python-sdk.md - Full SDK documentation - Installation instructions - Quick start examples - API reference - LiteLLM compatibility - Development setup - Publishing guide - tests/smoke_test.py - Smoke tests - Import test - completion() test - acompletion() test - embedding() test - aembedding() test - Exceptions test - LiteLLM alias test - .github/workflows/quota-router-python.yml - CI workflow - Build and test - Type checking - Wheel building
Mark completed missions: - 0908-a: All criteria completed (PyO3, async, type stubs) - 0908-c: Embedding functions completed Mark partial progress: - 0908-d: pyproject.toml, package, CI done - 0908-e: Core extracted, functions via PyO3 done
Add #![allow(deprecated)] to suppress pyo3 deprecation warnings for PyDict::new and PyList::new which will be replaced in future versions. Also run cargo fmt to fix formatting.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request Checklist
Type of Change
agent/*branch)Branch Strategy
This PR follows the CipherOcto branch strategy:
feat/*nextagent/*nextresearch/*nexthotfix/*mainnextmainCurrent PR:
<!-- source branch -->→<!-- target branch -->Description
Related Issues
Closes #(issue)
Testing
Performance Impact
Security Considerations
Additional Notes