Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [0.1.0] - 2025-03-09

### Added

- Google style docstring parsing (`parse_google`)
- NumPy style docstring parsing (`parse_numpy`)
- Automatic style detection (`detect_style`)
- Unified model IR (`Docstring`, `Section`, `Parameter`, `Return`, etc.)
- Emit back to Google style (`emit_google`)
- Emit back to NumPy style (`emit_numpy`)
- Full syntax tree (AST) with byte-precise source locations (`TextRange`)
- Tree traversal via `walk` and visitor pattern
- Pretty-print for AST debugging (`pretty_print`)
- Conversion from AST to unified model (`to_model`)
- Support for all standard sections:
- Parameters / Args / Keyword Args / Other Parameters
- Returns / Yields
- Raises / Warns
- Attributes / Methods
- See Also / References
- Deprecation
- Free-text sections (Notes, Examples, Warnings, Todo, etc.)
- Error-resilient parsing — never panics on malformed input
- Zero external crate dependencies
- Python bindings via PyO3 (`pydocstring-rs`)

[0.1.0]: https://github.com/qraqras/pydocstring/releases/tag/v0.1.0
2 changes: 1 addition & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "pydocstring"
version = "0.0.3"
version = "0.1.0"
edition = "2024"
authors = ["Ryuma Asai"]
description = "A zero-dependency Rust parser for Python docstrings (Google and NumPy styles) with a unified syntax tree and byte-precise source locations"
Expand Down
75 changes: 66 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,16 +24,15 @@ Python bindings are also available as [`pydocstring-rs`](https://pypi.org/projec

```toml
[dependencies]
pydocstring = "0.0.3"
pydocstring = "0.1.0"
```

## Usage

### Parsing

```rust
use pydocstring::google::{parse_google, nodes::GoogleDocstring};
use pydocstring::GoogleSectionKind;
use pydocstring::parse::google::{parse_google, GoogleDocstring, GoogleSectionKind};

let input = "Summary.\n\nArgs:\n x (int): The value.\n y (int): Another value.";
let result = parse_google(input);
Expand All @@ -57,7 +56,7 @@ NumPy style works the same way — use `parse_numpy` / `NumPyDocstring` instead.
### Style Auto-Detection

```rust
use pydocstring::{detect_style, Style};
use pydocstring::parse::{detect_style, Style};

assert_eq!(detect_style("Summary.\n\nArgs:\n x: Desc."), Style::Google);
assert_eq!(detect_style("Summary.\n\nParameters\n----------\nx : int"), Style::NumPy);
Expand All @@ -68,8 +67,7 @@ assert_eq!(detect_style("Summary.\n\nParameters\n----------\nx : int"), Style::N
Every token carries byte offsets for precise diagnostics:

```rust
use pydocstring::google::{parse_google, nodes::GoogleDocstring};
use pydocstring::GoogleSectionKind;
use pydocstring::parse::google::{parse_google, GoogleDocstring, GoogleSectionKind};

let result = parse_google("Summary.\n\nArgs:\n x (int): The value.");
let doc = GoogleDocstring::cast(result.root()).unwrap();
Expand All @@ -90,7 +88,7 @@ for section in doc.sections() {
The parse result is a tree of `SyntaxNode` (branches) and `SyntaxToken` (leaves), each tagged with a `SyntaxKind`. Use `pretty_print()` to visualize:

```rust
use pydocstring::google::parse_google;
use pydocstring::parse::google::parse_google;

let result = parse_google("Summary.\n\nArgs:\n x (int): The value.");
println!("{}", result.pretty_print());
Expand Down Expand Up @@ -121,8 +119,8 @@ GOOGLE_DOCSTRING@0..42 {
Walk the tree with the `Visitor` trait for style-agnostic analysis:

```rust
use pydocstring::{Visitor, walk, SyntaxToken, SyntaxKind};
use pydocstring::google::parse_google;
use pydocstring::syntax::{Visitor, walk, SyntaxToken, SyntaxKind};
use pydocstring::parse::google::parse_google;

struct NameCollector<'a> {
source: &'a str,
Expand All @@ -143,6 +141,65 @@ walk(result.root(), &mut collector);
assert_eq!(collector.names, vec!["Args", "x", "y"]);
```

### Style-Independent Model (IR)

Convert any parsed docstring into a style-independent intermediate representation for analysis or transformation:

```rust
use pydocstring::parse::google::{parse_google, to_model::to_model};

let parsed = parse_google("Summary.\n\nArgs:\n x (int): The value.\n");
let doc = to_model(&parsed).unwrap();

assert_eq!(doc.summary.as_deref(), Some("Summary."));
for section in &doc.sections {
if let pydocstring::model::Section::Parameters(params) = section {
assert_eq!(params[0].names, vec!["x"]);
assert_eq!(params[0].type_annotation.as_deref(), Some("int"));
}
}
```

### Emitting (Code Generation)

Re-emit a `Docstring` model in any style — useful for style conversion or formatting:

```rust
use pydocstring::model::{Docstring, Section, Parameter};
use pydocstring::emit::google::emit_google;
use pydocstring::emit::numpy::emit_numpy;

let doc = Docstring {
summary: Some("Brief summary.".into()),
sections: vec![Section::Parameters(vec![Parameter {
names: vec!["x".into()],
type_annotation: Some("int".into()),
description: Some("The value.".into()),
is_optional: false,
default_value: None,
}])],
..Default::default()
};

let google = emit_google(&doc);
assert!(google.contains("Args:"));

let numpy = emit_numpy(&doc);
assert!(numpy.contains("Parameters\n----------"));
```

Combine parsing and emitting to convert between styles:

```rust
use pydocstring::parse::google::{parse_google, to_model::to_model};
use pydocstring::emit::numpy::emit_numpy;

let parsed = parse_google("Summary.\n\nArgs:\n x (int): The value.\n");
let doc = to_model(&parsed).unwrap();
let numpy_text = emit_numpy(&doc);
assert!(numpy_text.contains("Parameters\n----------"));
```

## Supported Sections

Both styles support the following section categories. Typed accessor methods are available on each style's section node.
Expand Down
4 changes: 2 additions & 2 deletions bindings/python/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions bindings/python/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "pydocstring-python"
version = "0.0.3"
version = "0.1.0"
edition = "2024"
authors = ["Ryuma Asai"]
description = "Python bindings for pydocstring — a fast docstring parser for Google and NumPy styles"
Expand All @@ -12,5 +12,5 @@ name = "pydocstring"
crate-type = ["cdylib"]

[dependencies]
pydocstring_core = { package = "pydocstring", version = "0.0.3", path = "../.." }
pydocstring_core = { package = "pydocstring", version = "0.1.0", path = "../.." }
pyo3 = { version = "0.24", features = ["extension-module"] }
115 changes: 94 additions & 21 deletions bindings/python/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -165,34 +165,107 @@ token = doc.summary
print(token.range.start, token.range.end) # 0 8
```

### Style-Independent Model (IR)

Convert any parsed docstring into a style-independent intermediate representation for analysis or transformation:

```python
from pydocstring import parse_google

parsed = parse_google("Summary.\n\nArgs:\n x (int): The value.\n")
doc = parsed.to_model()

print(doc.summary) # "Summary."

for section in doc.sections:
if section.kind == "Parameters":
for param in section.parameters:
print(param.names) # ["x"]
print(param.type_annotation) # "int"
print(param.description) # "The value."
```

### Emitting (Code Generation)

Re-emit a `Docstring` model in any style — useful for style conversion or formatting:

```python
from pydocstring import Docstring, Section, Parameter, emit_google, emit_numpy

doc = Docstring(
summary="Brief summary.",
sections=[
Section(
"Parameters",
parameters=[
Parameter(
["x"],
type_annotation="int",
description="The value.",
),
],
),
],
)

google = emit_google(doc)
print(google) # Contains "Args:"

numpy = emit_numpy(doc)
print(numpy) # Contains "Parameters\n----------"
```

Combine parsing and emitting to convert between styles:

```python
from pydocstring import parse_google, emit_numpy

parsed = parse_google("Summary.\n\nArgs:\n x (int): The value.\n")
doc = parsed.to_model()
numpy_text = emit_numpy(doc)
print(numpy_text) # Contains "Parameters\n----------"
```

## API Reference

### Functions

| Function | Returns | Description |
|----------------------|-------------------|-----------------------------------------------|
| `parse_google(text)` | `GoogleDocstring` | Parse a Google-style docstring |
| `parse_numpy(text)` | `NumPyDocstring` | Parse a NumPy-style docstring |
| `detect_style(text)` | `Style` | Detect style: `Style.GOOGLE` or `Style.NUMPY` |
| Function | Returns | Description |
|----------------------|-------------------|------------------------------------------------|
| `parse_google(text)` | `GoogleDocstring` | Parse a Google-style docstring |
| `parse_numpy(text)` | `NumPyDocstring` | Parse a NumPy-style docstring |
| `detect_style(text)` | `Style` | Detect style: `Style.GOOGLE` or `Style.NUMPY` |
| `emit_google(doc)` | `str` | Emit a `Docstring` model as Google-style text |
| `emit_numpy(doc)` | `str` | Emit a `Docstring` model as NumPy-style text |

### Objects

| Class | Key Properties |
|-------------------|-------------------------------------------------------------------------------|
| `Style` | `GOOGLE`, `NUMPY` (enum) |
| `GoogleDocstring` | `summary`, `extended_summary`, `sections`, `node`, `source`, `pretty_print()` |
| `GoogleSection` | `kind`, `args`, `returns`, `exceptions`, `body_text`, `node` |
| `GoogleArg` | `name`, `type`, `description`, `optional` |
| `GoogleReturns` | `return_type`, `description` |
| `GoogleException` | `type`, `description` |
| `NumPyDocstring` | `summary`, `extended_summary`, `sections`, `node`, `source`, `pretty_print()` |
| `NumPySection` | `kind`, `parameters`, `returns`, `exceptions`, `body_text`, `node` |
| `NumPyParameter` | `names`, `type`, `description`, `optional`, `default_value` |
| `NumPyReturns` | `name`, `return_type`, `description` |
| `NumPyException` | `type`, `description` |
| `Token` | `kind`, `text`, `range` |
| `Node` | `kind`, `range`, `children` |
| `TextRange` | `start`, `end` |
| Class | Key Properties |
|-------------------|------------------------------------------------------------------------------------------------------------------|
| `Style` | `GOOGLE`, `NUMPY` (enum) |
| `GoogleDocstring` | `summary`, `extended_summary`, `sections`, `node`, `source`, `pretty_print()`, `to_model()` |
| `GoogleSection` | `kind`, `args`, `returns`, `exceptions`, `body_text`, `node` |
| `GoogleArg` | `name`, `type`, `description`, `optional` |
| `GoogleReturns` | `return_type`, `description` |
| `GoogleException` | `type`, `description` |
| `NumPyDocstring` | `summary`, `extended_summary`, `sections`, `node`, `source`, `pretty_print()`, `to_model()` |
| `NumPySection` | `kind`, `parameters`, `returns`, `exceptions`, `body_text`, `node` |
| `NumPyParameter` | `names`, `type`, `description`, `optional`, `default_value` |
| `NumPyReturns` | `name`, `return_type`, `description` |
| `NumPyException` | `type`, `description` |
| `Token` | `kind`, `text`, `range` |
| `Node` | `kind`, `range`, `children` |
| `TextRange` | `start`, `end` |
| `Docstring` | `summary`, `extended_summary`, `deprecation`, `sections` |
| `Section` (model) | `kind`, `parameters`, `returns`, `exceptions`, `attributes`, `methods`, `see_also_entries`, `references`, `body` |
| `Parameter` | `names`, `type_annotation`, `description`, `is_optional`, `default_value` |
| `Return` | `name`, `type_annotation`, `description` |
| `ExceptionEntry` | `type_name`, `description` |
| `Attribute` | `name`, `type_annotation`, `description` |
| `Method` | `name`, `type_annotation`, `description` |
| `SeeAlsoEntry` | `names`, `description` |
| `Reference` | `number`, `content` |
| `Deprecation` | `version`, `description` |

## Development

Expand Down
Loading