chore: Automatically generate example format tabs by bradenhilton · Pull Request #4959 · twpayne/chezmoi

bradenhilton · 2026-03-15T02:00:01Z

Fixes #4954

bradenhilton · 2026-03-16T00:46:34Z

Just to clarify, I'm waiting for a bit of feedback on this (whenever anyone has time) before I proceed any further.

As always, there's no rush.

twpayne

This looks excellent to me. I particularly like that this is done inside the material-mkdocs framework.

Another nice feature is that when you choose the format, all examples in the same page change to use that format.

Great work @bradenhilton!

bradenhilton · 2026-03-16T11:52:39Z

I've decided to not convert any blocks that contain template syntax, since tomllib cannot parse them as is.

It's non-trivial to manipulate the TOML text to work around this if the TOML is comprised of sections with template syntax between them, or if it contains sequences like {{ ... | toToml }}. One particular problem area would be any examples of templated externals.

At a minimum, you would probably need to do something like:

Split the text on template lines, so you get each TOML section separately
Store the template lines somewhere, along with some kind of index so you know where they were originally
Parse and convert each TOML section individually
Put the template lines back in-between each section
(optional) Use a ```text fence instead to prevent ugly highlighting on the syntax "errors"

(there's probably a smarter/more elegant way to do this (perhaps a way to do it in a single pass or something), but the idea is essentially the same)

It probably can be solved, but in these cases it's probably far easier to just manually write the examples, especially now that the basic examples are done automatically.

Also, parsing most TOML blocks acts like a linter of sorts. I actually found a subtle quoting error on one of the blocks because it wouldn't parse correctly.

KapJI · 2026-03-16T11:56:44Z

It can enforce manual conversions if it can't be done automatically, so we can make sure all examples are available in all versions.

bradenhilton · 2026-03-16T21:04:24Z

I've made a first pass at changing examples.

I searched the site directory for ```toml to find them.

I also added some temporary code that outputs all modified markdown files to an examples_preview directory at the root, for convenience in reviewing. I'll remove it before merging.

bradenhilton · 2026-03-18T10:35:36Z

+    # We allow capturing *.toml.tmpl filename titles but do not support
+    # converting examples containing any template syntax.


I should probably update this to clarify that things like template syntax within values is fine, but at the moment we don't support things like template syntax around or in-between blocks of TOML

KapJI · 2026-03-22T21:53:06Z

+        return textwrap.indent(examples, indent)
+
+    new_md = example_pattern.sub(replace, markdown)
+    if new_md != markdown:


This can be useful for debugging, hide behind env var?

I don't disagree, but it's outside the scope of this PR, and temporary environment variables don't have great ergonomics on Windows.

Perhaps we can integrate it into taskipy (or some other task runner), but I won't add it to this PR.

cc @halostatue, who is more familiar with taskipy

I'd have to investigate.

We could maybe look into replacing Make+taskipy with Just or Task at some point

Not sure I get your "outside the scope" argument. This is debug feature which needs to be disabled or removed before merging. I think it makes sense to keep it and activate by CLI flag (e.g. --debug) or env var.

Adding this feature will likely require using an env var directly inside a task runner, as I alluded to in #4959 (comment). The feature is also potentially useful for other kinds of transformations, not just the one specific to this PR.

If it can't be done with taskipy, that means we need to potentially switch to a different task runner (if we can find one that works). I don't want to do either that or manual handling of an env var in this PR because I am currently using Windows, and temporary env vars for a single command are beyond tedious unless you use a sane operating system.

The debug output code is temporary and will be removed before this PR is merged.

halostatue · 2026-03-24T16:09:47Z

+        return textwrap.indent(examples, indent)
+
+    new_md = example_pattern.sub(replace, markdown)
+    if new_md != markdown:


I'd have to investigate.

KapJI · 2026-03-27T21:39:24Z

+        yaml_obj.dump(data, yaml_stream)
+        yaml_text = yaml_stream.getvalue()
+
+    json_text = json.dumps(data, indent=4)


tomllib.loads may return non-serializable type:

text = tomllib.loads("updated = 2026-03-27") json.dumps(text) # TypeError: Object of type datetime is not JSON serializable

It'd be nice to handle such error.

The TypeError is sufficient, no?

The primary point of this change is to make it easier and less error-prone to provide equivalent examples in multiple formats. If any example contains any types that are not serializable in a different format, it isn't suitable for this transformation.

Tom also said it's fine for errors to blow up in #4959 (comment).

KapJI · 2026-03-27T21:40:51Z

+                elif FENCE_CLOSE_RX.match(line):
+                    state = MarkdownTransformState.IN_BLOCK
+                else:
+                    if line.lstrip().startswith('{{'):


Can template be in the middle of the line?

I believe only template syntax within strings will parse correctly (and also work in the other formats). I am aware that the current approach is naive.

Perhaps it would be better to pull the TOML parsing/validation out into a separate function that is called in the loop, rather than doing it inside the function that also creates the tabs.

KapJI · 2026-03-27T21:43:10Z

+        return textwrap.indent(examples, indent)
+
+    new_md = example_pattern.sub(replace, markdown)
+    if new_md != markdown:


Not sure I get your "outside the scope" argument. This is debug feature which needs to be disabled or removed before merging. I think it makes sense to keep it and activate by CLI flag (e.g. --debug) or env var.

KapJI · 2026-03-27T21:46:22Z

+def on_page_markdown(markdown: str, page: Page, **kwargs) -> str:
+    lines = markdown.splitlines()
+
+    new_md = '\n'.join(transform_markdown(lines))


It'd be good to add a training newline here.

KapJI · 2026-03-27T21:49:07Z

 ]


+class MarkdownTransformState(Enum):


This is much cleaner than old regexp!

twpayne

This is excellent, thank you very much @bradenhilton for the work and @KapJI and @halostatue for the reviews.

One minor thing I noticed is that example configs don't get highlighted correctly when they're in !!! info boxes, but this is almost certainly an issue with mkdocs-material and doesn't need to be fixed here. The easy fix is probably to simply avoid using so many !!! info boxes.

Please tidy up the commit history and merge it when you feel that it's ready.

bradenhilton force-pushed the examples branch from cb44648 to 44aac23 Compare March 15, 2026 02:02

twpayne reviewed Mar 16, 2026

View reviewed changes

Comment thread assets/chezmoi.io/docs/hooks.py Outdated

Comment thread assets/chezmoi.io/docs/hooks.py Outdated

twpayne reviewed Mar 16, 2026

View reviewed changes

Comment thread assets/chezmoi.io/mkdocs.yml

bradenhilton force-pushed the examples branch from 5a25fd2 to 509cd76 Compare March 16, 2026 21:02

bradenhilton requested review from halostatue and twpayne March 16, 2026 21:06

bradenhilton force-pushed the examples branch from 5036394 to 2c9e8e6 Compare March 17, 2026 00:08

bradenhilton commented Mar 18, 2026

View reviewed changes

KapJI reviewed Mar 22, 2026

View reviewed changes

Comment thread assets/chezmoi.io/docs/user-guide/templating.md Outdated

Comment thread assets/chezmoi.io/docs/user-guide/include-files-from-elsewhere.md Outdated

KapJI reviewed Mar 22, 2026

View reviewed changes

Comment thread assets/chezmoi.io/docs/hooks.py Outdated

KapJI reviewed Mar 22, 2026

View reviewed changes

halostatue reviewed Mar 24, 2026

View reviewed changes

KapJI reviewed Mar 27, 2026

View reviewed changes

twpayne approved these changes Mar 29, 2026

View reviewed changes

bradenhilton force-pushed the examples branch from b7b8418 to edd2591 Compare April 10, 2026 15:02

chore: Automatically generate example format tabs

50c8b73

bradenhilton force-pushed the examples branch from edd2591 to 50c8b73 Compare April 10, 2026 15:59

bradenhilton merged commit d844fad into twpayne:master Apr 10, 2026
20 checks passed

bradenhilton deleted the examples branch April 14, 2026 17:12

		# We allow capturing *.toml.tmpl filename titles but do not support
		# converting examples containing any template syntax.

		]


		class MarkdownTransformState(Enum):

Conversation

bradenhilton commented Mar 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bradenhilton commented Mar 16, 2026

Uh oh!

twpayne left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bradenhilton commented Mar 16, 2026

Uh oh!

KapJI commented Mar 16, 2026

Uh oh!

bradenhilton commented Mar 16, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

twpayne left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

bradenhilton commented Mar 15, 2026 •

edited

Loading