|
| 1 | +# Rust Extensions |
| 2 | + |
| 3 | +This page explains how to add your own async Rust code and use it alongside |
| 4 | +`rsloop` in Python. |
| 5 | + |
| 6 | +The idea is: |
| 7 | + |
| 8 | +- Python still uses `rsloop` as the event loop |
| 9 | +- your own PyO3 extension module exposes extra functions |
| 10 | +- those functions return Python awaitables backed by Rust futures |
| 11 | + |
| 12 | +`rsloop` provides a small Rust helper module for this: |
| 13 | + |
| 14 | +- `rsloop::rust_async::get_current_locals(...)` |
| 15 | +- `rsloop::rust_async::future_into_py(...)` |
| 16 | +- `rsloop::rust_async::future_into_py_with_locals(...)` |
| 17 | +- `rsloop::rust_async::local_future_into_py(...)` |
| 18 | +- `rsloop::rust_async::local_future_into_py_with_locals(...)` |
| 19 | +- `rsloop::rust_async::TaskLocals` |
| 20 | +- `rsloop::rust_async::into_future_with_locals(...)` |
| 21 | + |
| 22 | +## When to use this |
| 23 | + |
| 24 | +Use this pattern when: |
| 25 | + |
| 26 | +- you want to keep the main application in Python |
| 27 | +- you want some operations implemented in Rust |
| 28 | +- those Rust operations should be `await`-able from Python |
| 29 | +- you want them to run under the same active `rsloop` loop |
| 30 | + |
| 31 | +This is not for modifying `rsloop` itself. It is for building a separate Rust |
| 32 | +extension crate that depends on `rsloop`. |
| 33 | + |
| 34 | +## What the developer builds |
| 35 | + |
| 36 | +You usually create a second crate with: |
| 37 | + |
| 38 | +1. a `Cargo.toml` for your PyO3 extension |
| 39 | +2. a `pyproject.toml` for `maturin` |
| 40 | +3. Rust functions marked with `#[pyfunction]` |
| 41 | +4. a `#[pymodule]` that exports those functions |
| 42 | + |
| 43 | +The important part is that each async Rust function should return a Python |
| 44 | +awaitable using `rsloop::rust_async::future_into_py(...)`. |
| 45 | + |
| 46 | +## Minimal Rust example |
| 47 | + |
| 48 | +```rust |
| 49 | +use std::time::Duration; |
| 50 | + |
| 51 | +use pyo3::prelude::*; |
| 52 | + |
| 53 | +#[pyfunction] |
| 54 | +fn sleep_and_tag(py: Python<'_>, label: String, delay_ms: u64) -> PyResult<Bound<'_, PyAny>> { |
| 55 | + rsloop::rust_async::future_into_py(py, async move { |
| 56 | + async_std::task::sleep(Duration::from_millis(delay_ms)).await; |
| 57 | + Ok(format!("rust finished: {label}")) |
| 58 | + }) |
| 59 | +} |
| 60 | + |
| 61 | +#[pymodule(gil_used = false)] |
| 62 | +fn my_rust_ext(m: &Bound<'_, PyModule>) -> PyResult<()> { |
| 63 | + m.add_function(wrap_pyfunction!(sleep_and_tag, m)?)?; |
| 64 | + Ok(()) |
| 65 | +} |
| 66 | +``` |
| 67 | + |
| 68 | +In this example: |
| 69 | + |
| 70 | +- the Python caller gets an awaitable back immediately |
| 71 | +- the real work happens in the Rust future |
| 72 | +- the future is attached to the currently running Python event loop |
| 73 | + |
| 74 | +## Python side usage |
| 75 | + |
| 76 | +From Python, the Rust function looks like a normal async operation: |
| 77 | + |
| 78 | +```python |
| 79 | +import asyncio |
| 80 | +import rsloop |
| 81 | +import my_rust_ext |
| 82 | + |
| 83 | + |
| 84 | +async def main() -> None: |
| 85 | + loop = asyncio.get_running_loop() |
| 86 | + print(type(loop).__name__) |
| 87 | + |
| 88 | + result = await my_rust_ext.sleep_and_tag("hello from python", 50) |
| 89 | + print(result) |
| 90 | + |
| 91 | + |
| 92 | +rsloop.run(main()) |
| 93 | +``` |
| 94 | + |
| 95 | +This works because `rsloop.run(...)` creates the running loop first, and then |
| 96 | +the Rust extension captures that loop when `future_into_py(...)` is called. |
| 97 | + |
| 98 | +## Project setup |
| 99 | + |
| 100 | +Your extension crate should depend on: |
| 101 | + |
| 102 | +```toml |
| 103 | +[dependencies] |
| 104 | +async-std = "1" |
| 105 | +pyo3 = { version = "0.28", features = ["extension-module"] } |
| 106 | +rsloop = { path = "/path/to/rsloop" } |
| 107 | +``` |
| 108 | + |
| 109 | +For a real package, replace the path dependency with the form that matches your |
| 110 | +project layout. |
| 111 | + |
| 112 | +Your `pyproject.toml` should use `maturin` in the normal PyO3 way: |
| 113 | + |
| 114 | +```toml |
| 115 | +[build-system] |
| 116 | +requires = ["maturin>=1.7,<2"] |
| 117 | +build-backend = "maturin" |
| 118 | + |
| 119 | +[tool.maturin] |
| 120 | +module-name = "my_rust_ext" |
| 121 | +``` |
| 122 | + |
| 123 | +## How the bridge works |
| 124 | + |
| 125 | +`future_into_py(...)` does two important things: |
| 126 | + |
| 127 | +1. It captures the current Python event loop and contextvars. |
| 128 | +2. It converts the Rust future into a Python awaitable. |
| 129 | + |
| 130 | +That means your Rust future can be awaited from Python code that is already |
| 131 | +running on `rsloop`. |
| 132 | + |
| 133 | +If you need lower-level control, use `get_current_locals(...)` and |
| 134 | +`future_into_py_with_locals(...)` directly. That is useful when you capture the |
| 135 | +loop/context once and reuse it across multiple Rust operations. |
| 136 | + |
| 137 | +## Calling Python awaitables from Rust |
| 138 | + |
| 139 | +`rsloop` also re-exports `into_future_with_locals(...)`. |
| 140 | + |
| 141 | +Use that when your Rust future needs to await a Python awaitable: |
| 142 | + |
| 143 | +```rust |
| 144 | +use pyo3::prelude::*; |
| 145 | + |
| 146 | +fn call_python_awaitable( |
| 147 | + py: Python<'_>, |
| 148 | + awaitable: Py<PyAny>, |
| 149 | +) -> PyResult<Bound<'_, PyAny>> { |
| 150 | + let locals = rsloop::rust_async::get_current_locals(py)?; |
| 151 | + rsloop::rust_async::future_into_py_with_locals(py, locals.clone(), async move { |
| 152 | + let python_future = Python::attach(|py| { |
| 153 | + rsloop::rust_async::into_future_with_locals(&locals, awaitable.bind(py).clone()) |
| 154 | + })?; |
| 155 | + let value = python_future.await?; |
| 156 | + Ok(value) |
| 157 | + }) |
| 158 | +} |
| 159 | +``` |
| 160 | + |
| 161 | +That pattern is more advanced, but it is useful when Rust is coordinating both |
| 162 | +Rust futures and Python coroutines. |
| 163 | + |
| 164 | +## Full example in this repository |
| 165 | + |
| 166 | +The repository includes a complete example in: |
| 167 | + |
| 168 | +- `examples/rust/src/lib.rs` |
| 169 | +- `examples/rust/demo.py` |
| 170 | +- `examples/rust/README.md` |
| 171 | + |
| 172 | +If you want the shortest end-to-end check from the repository root, run: |
| 173 | + |
| 174 | +```bash |
| 175 | +uv run --with . --with ./examples/rust python examples/rust/demo.py |
| 176 | +``` |
| 177 | + |
| 178 | +## Practical advice |
| 179 | + |
| 180 | +- Keep the Rust extension separate from the `rsloop` crate itself. |
| 181 | +- Start with small `#[pyfunction]` wrappers that return one awaitable each. |
| 182 | +- Prefer returning normal Python-friendly values such as strings, integers, dictionaries, and lists. |
| 183 | +- Use `future_into_py(...)` first. Drop to the `*_with_locals(...)` variants only when you actually need them. |
| 184 | +- Make sure your Rust function is called while a Python event loop is already running. |
| 185 | + |
| 186 | +If you call the helper outside a running event loop, capturing the current loop |
| 187 | +will fail, because there is no active Python loop to attach to. |
0 commit comments