Skip to content

Commit 6ce6ed4

Browse files
committed
add ability to create rust code that works with asyncio code
closes #34
1 parent 7b6364e commit 6ce6ed4

16 files changed

Lines changed: 2061 additions & 4 deletions

Cargo.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ homepage = "https://github.com/RustedBytes/rsloop"
1010
documentation = "https://github.com/RustedBytes/rsloop#readme"
1111

1212
[lib]
13-
name = "_loop"
13+
name = "rsloop"
1414
crate-type = ["cdylib", "rlib"]
1515

1616
[features]

README.md

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -88,6 +88,24 @@ Importing `rsloop` also patches `asyncio.set_event_loop()` so Python 3.8 can
8888
accept an `rsloop.Loop` instance, matching the behavior exercised by
8989
[`tests/test_run.py`](./tests/test_run.py).
9090

91+
## Custom Async Rust Extensions
92+
93+
`rsloop` now exposes a small Rust interop API for downstream PyO3 extensions.
94+
That lets you write your own async Rust code, return it to Python as an
95+
awaitable, and run it under the active `rsloop` event loop.
96+
97+
The public entry point is `rsloop::rust_async`:
98+
99+
- `get_current_locals(...)`
100+
- `future_into_py(...)`
101+
- `future_into_py_with_locals(...)`
102+
- `local_future_into_py(...)`
103+
- `local_future_into_py_with_locals(...)`
104+
- re-exports of `TaskLocals` and `into_future_with_locals(...)`
105+
106+
See [`examples/rust/README.md`](./examples/rust/README.md) for a complete
107+
extension example built with `maturin`.
108+
91109
## Verified Surface Area
92110

93111
The current codebase implements these user-facing areas.

docs/getting-started.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -107,3 +107,11 @@ The `examples/` directory is the best hands-on tour of the project:
107107
If you are new to lower-level `asyncio` features, start with `01_basics.py` and `03_streams.py`.
108108

109109
There is also a shorter docs page with copy-paste snippets in [Examples](examples.md).
110+
111+
## Adding your own async Rust code
112+
113+
If you want to keep using `rsloop` in Python while exposing your own async Rust
114+
functions from a separate PyO3 extension, read [Rust Extensions](rust-extensions.md).
115+
116+
That page explains how to turn a Rust future into a Python awaitable with
117+
`rsloop::rust_async::future_into_py(...)`.

docs/how-it-works.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -54,6 +54,7 @@ Important files:
5454
- `context.rs`: running-loop and context management helpers
5555
- `errors.rs`: shared error types
5656
- `profiler.rs`: Tracy profiler support
57+
- `rust_async.rs`: public Rust/Python async interop helpers for downstream extensions
5758
- `async_event.rs`, `blocking.rs`, `python_names.rs`: support code used by the public pieces
5859
- `windows_vibeio.rs`: Windows-specific support
5960

docs/index.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,7 @@ rsloop.run(main())
7474
## Recommended reading order
7575

7676
- Start with [Getting Started](getting-started.md) if you want to use the package.
77+
- Read [Rust Extensions](rust-extensions.md) if you want to add your own async Rust functions and await them from Python.
7778
- Read [Examples](examples.md) if you want copy-paste usage patterns.
7879
- Read [How It Works](how-it-works.md) if you want the big picture.
7980
- Read [Project Structure](project-structure.md) if you want to explore the codebase.

docs/rust-extensions.md

Lines changed: 187 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,187 @@
1+
# Rust Extensions
2+
3+
This page explains how to add your own async Rust code and use it alongside
4+
`rsloop` in Python.
5+
6+
The idea is:
7+
8+
- Python still uses `rsloop` as the event loop
9+
- your own PyO3 extension module exposes extra functions
10+
- those functions return Python awaitables backed by Rust futures
11+
12+
`rsloop` provides a small Rust helper module for this:
13+
14+
- `rsloop::rust_async::get_current_locals(...)`
15+
- `rsloop::rust_async::future_into_py(...)`
16+
- `rsloop::rust_async::future_into_py_with_locals(...)`
17+
- `rsloop::rust_async::local_future_into_py(...)`
18+
- `rsloop::rust_async::local_future_into_py_with_locals(...)`
19+
- `rsloop::rust_async::TaskLocals`
20+
- `rsloop::rust_async::into_future_with_locals(...)`
21+
22+
## When to use this
23+
24+
Use this pattern when:
25+
26+
- you want to keep the main application in Python
27+
- you want some operations implemented in Rust
28+
- those Rust operations should be `await`-able from Python
29+
- you want them to run under the same active `rsloop` loop
30+
31+
This is not for modifying `rsloop` itself. It is for building a separate Rust
32+
extension crate that depends on `rsloop`.
33+
34+
## What the developer builds
35+
36+
You usually create a second crate with:
37+
38+
1. a `Cargo.toml` for your PyO3 extension
39+
2. a `pyproject.toml` for `maturin`
40+
3. Rust functions marked with `#[pyfunction]`
41+
4. a `#[pymodule]` that exports those functions
42+
43+
The important part is that each async Rust function should return a Python
44+
awaitable using `rsloop::rust_async::future_into_py(...)`.
45+
46+
## Minimal Rust example
47+
48+
```rust
49+
use std::time::Duration;
50+
51+
use pyo3::prelude::*;
52+
53+
#[pyfunction]
54+
fn sleep_and_tag(py: Python<'_>, label: String, delay_ms: u64) -> PyResult<Bound<'_, PyAny>> {
55+
rsloop::rust_async::future_into_py(py, async move {
56+
async_std::task::sleep(Duration::from_millis(delay_ms)).await;
57+
Ok(format!("rust finished: {label}"))
58+
})
59+
}
60+
61+
#[pymodule(gil_used = false)]
62+
fn my_rust_ext(m: &Bound<'_, PyModule>) -> PyResult<()> {
63+
m.add_function(wrap_pyfunction!(sleep_and_tag, m)?)?;
64+
Ok(())
65+
}
66+
```
67+
68+
In this example:
69+
70+
- the Python caller gets an awaitable back immediately
71+
- the real work happens in the Rust future
72+
- the future is attached to the currently running Python event loop
73+
74+
## Python side usage
75+
76+
From Python, the Rust function looks like a normal async operation:
77+
78+
```python
79+
import asyncio
80+
import rsloop
81+
import my_rust_ext
82+
83+
84+
async def main() -> None:
85+
loop = asyncio.get_running_loop()
86+
print(type(loop).__name__)
87+
88+
result = await my_rust_ext.sleep_and_tag("hello from python", 50)
89+
print(result)
90+
91+
92+
rsloop.run(main())
93+
```
94+
95+
This works because `rsloop.run(...)` creates the running loop first, and then
96+
the Rust extension captures that loop when `future_into_py(...)` is called.
97+
98+
## Project setup
99+
100+
Your extension crate should depend on:
101+
102+
```toml
103+
[dependencies]
104+
async-std = "1"
105+
pyo3 = { version = "0.28", features = ["extension-module"] }
106+
rsloop = { path = "/path/to/rsloop" }
107+
```
108+
109+
For a real package, replace the path dependency with the form that matches your
110+
project layout.
111+
112+
Your `pyproject.toml` should use `maturin` in the normal PyO3 way:
113+
114+
```toml
115+
[build-system]
116+
requires = ["maturin>=1.7,<2"]
117+
build-backend = "maturin"
118+
119+
[tool.maturin]
120+
module-name = "my_rust_ext"
121+
```
122+
123+
## How the bridge works
124+
125+
`future_into_py(...)` does two important things:
126+
127+
1. It captures the current Python event loop and contextvars.
128+
2. It converts the Rust future into a Python awaitable.
129+
130+
That means your Rust future can be awaited from Python code that is already
131+
running on `rsloop`.
132+
133+
If you need lower-level control, use `get_current_locals(...)` and
134+
`future_into_py_with_locals(...)` directly. That is useful when you capture the
135+
loop/context once and reuse it across multiple Rust operations.
136+
137+
## Calling Python awaitables from Rust
138+
139+
`rsloop` also re-exports `into_future_with_locals(...)`.
140+
141+
Use that when your Rust future needs to await a Python awaitable:
142+
143+
```rust
144+
use pyo3::prelude::*;
145+
146+
fn call_python_awaitable(
147+
py: Python<'_>,
148+
awaitable: Py<PyAny>,
149+
) -> PyResult<Bound<'_, PyAny>> {
150+
let locals = rsloop::rust_async::get_current_locals(py)?;
151+
rsloop::rust_async::future_into_py_with_locals(py, locals.clone(), async move {
152+
let python_future = Python::attach(|py| {
153+
rsloop::rust_async::into_future_with_locals(&locals, awaitable.bind(py).clone())
154+
})?;
155+
let value = python_future.await?;
156+
Ok(value)
157+
})
158+
}
159+
```
160+
161+
That pattern is more advanced, but it is useful when Rust is coordinating both
162+
Rust futures and Python coroutines.
163+
164+
## Full example in this repository
165+
166+
The repository includes a complete example in:
167+
168+
- `examples/rust/src/lib.rs`
169+
- `examples/rust/demo.py`
170+
- `examples/rust/README.md`
171+
172+
If you want the shortest end-to-end check from the repository root, run:
173+
174+
```bash
175+
uv run --with . --with ./examples/rust python examples/rust/demo.py
176+
```
177+
178+
## Practical advice
179+
180+
- Keep the Rust extension separate from the `rsloop` crate itself.
181+
- Start with small `#[pyfunction]` wrappers that return one awaitable each.
182+
- Prefer returning normal Python-friendly values such as strings, integers, dictionaries, and lists.
183+
- Use `future_into_py(...)` first. Drop to the `*_with_locals(...)` variants only when you actually need them.
184+
- Make sure your Rust function is called while a Python event loop is already running.
185+
186+
If you call the helper outside a running event loop, capturing the current loop
187+
will fail, because there is no active Python loop to attach to.

0 commit comments

Comments
 (0)