basic-text

Basic Text strings and I/O streams

This repository hosts a project to define a text format called Basic Text, a subset of Unicode that focuses on text: it excludes unprintable control characters, characters which depend on out-of-band information to interpret, non-characters, deprecated characters, and more, while aiming to preserve everything of practical use to plain text and formats built on top of it. See the Book for more information.

It also includes a Rust crate which aims to implement the Basic Text format, providing several utilities:

• TextString and TextStr are similar to the standard library's String and str, but use the Basic Text string format, along with a text!("...") macro for Basic Text string literals.

• TextReader and TextWriter are input and output streams which use the Basic Text stream format. On input, content is converted in a way which is lossy with respect to the original bytestream. Output uses the "strict" conversion method, in which invalid content is diagnosed with errors.

• BufReadText, an extension trait that adds text_lines and text_lines_lossy to BufRead implementations for reading lines from an input stream as BasicText strings.

• TextDuplexer is a Duplex for reading and writing on an interactive stream using Basic Text.

The code here is usable, and implements everything in the Basic Text spec, but is not very mature or optimized yet.