I'm not understanding the glue between this generator and the syncthing service. How do the files created by this generator actually get wired up into the running service?

Hope I am not misreading your question, but the pem files are seemingly simply used to set conventional nixos options such as on line 639-641 in this example.

The device id in syncthing.pub does not seem to be used in the example, but only stored. Presumably for outside use just as scripted setups of other devices to sync with.

nit: I'd feel better about this if we somehow asserted that the result of this is exactly 1 ID. I imagine things would get weird if this started returning multiple IDs or no ID at all.

Or, perhaps we could ask upstream syncthing if they'd be open to a change to syncthing generate to persist the id somewhere?

should guarantee that it can yield no more than 1 id (uniq could still yield multiple, different ids). Due to the regex match not applying to empty strings, grep will already return rc 1 if no id is found, so the set -e in the reference backend should fetch it.

Might be a good idea to explicitly define that scripts are supposed to be run with set -e. :)

If I understand correctly, this script may be run on the deployer host, which can be a different platform.

It is then up to the caller to pass a suitable pkgs instance that works for the deployer host.

EDIT: this is of course not the complete solution because the script contents still refer to pkgs. Use a separate file and callPackage to avoid a footgun.

As a test case, you could add aarch64-darwin to this and run the checks on darwin.

usually we overwrite pkgs of the system where the vars are defined if the caller architecture is different from the target systems architecture, since this also supports string context references to pkgs and the vars definitons want to live next to the service definitions.

I recommend against that. It blurs the line too much.

• You'll be evaluating a NixOS configuration that "runs on" macOS, which is unexpected. It may work out for you anyway, thanks to laziness, but if vars calls someone's code that reads something from the rest of the NixOS config (accidentally or otherwise), they'll get strange errors like "util-linux is not supported on darwin" or worse.
• Overriding is a code smell.
• Overriding is in general less efficient than evaluating the right thing in one go, especially if it's a NixOS config.

Explicit is good. By making this a proper parameter, readers and module authors have at least an idea of what's going on, and you don't rely on the overriding magic to never hit them in the face.

(unless this some ultra-obscure bash syntax i am not aware of ;)

I sadly think that "Inconsistent state for generator: ${gen.name}" is a pretty bad error message here. I think it should at least try to communicate to the user what exactly is wrong, i.e. which file was found / not found. Ideally it would communicate something actionable as well, i.e:

You can try purging ''${OUT_DIR:-${cfg.fileLocation}}, after making sure it contains nothing you want to keep

I understand that it's annoying to implement in bash, but I am not sure that bash is the right choice of language here in the first place tbh.

We should slap umask 077 in places...

mktemp -d does the mkdiring.

This API smells a bit weird to me: if I'm reading this correctly, we expect promptCmd to be a snippet of shell code that writes to a $prompt_value variable. Why not make it some command that prints a result to stdout (and exits non zero if something went wrong)?

Does this need shell escaping? Could gen.name contain a double quote? I think so.

Not double quotes as currently all names have the type lib.types.strMatching "[a-zA-Z0-9:_\\.-]*", but ending a name with \ would break this line. Quoting using ' would alleviate this, but for robustness sake, escaping most injected values seems preferable to me.

Traps override each other. Confer (trap 'echo a' EXIT; trap 'echo b' EXIT).

The way it stands, unless the script exits prematurely, only the very final 'rm -rf $out' gets executed, leaving all the other $outs, and all $ins and $promptss unremoved. This issue becomes apparent when declaring variables readonly.

One possible solution would be to spawn a subshell for each generator, e.g.:

...
elif [ $all_files_missing = true ] ; then
  (
    prompts=$(mktemp -d)
    in=$(mktemp -d)
    out=$(mktemp -d)
    readonly prompts in out # optional assertion
    trap 'rm -fR "$prompts" "$in" "$out"' EXIT
    export prompts in out
    ...
  )
fi

It's also possible to have an array of cleanup function: https://github.com/Mic92/dotfiles/blob/4ce52e794985cf159e69e259e8b63d2f7cadfa22/pkgs/merge-when-green/script.sh#L10

Can we somehow reuse vars.settings.fileModule here? I see how we do want OUT_DIR to be overridable, but reconstructing this path it multiple times feels wrong to me.

nit: This is unnecessary because of the trap 'rm -rf $out' EXIT up above, right?

ah, the trap only runs on exit, it's better to clean up the files if they are no longer needed earlier. the trap is just in case the script exists unexpectedly

nit: This is a weird name for me. It sounds like it's for the location of a file, but it's actually for the location of a directory of files. Perhaps varsLocation or varFilesLocation?

Not RFC 42. What's the thought behind this name?

I understand it's the provider-specific settings, similar in spirit to how SelfHostBlocks does it, except here it's (IMO correctly) in terms of modules.

Maybe

This contains both generators (executables), as well as their target files (calls of those executables).
Since those are in a 1:many relationship (potentially), this part of the option tree should be named after the "target files" instead (or whichever term those are described with).

It seems that you have an insofar unnamed domain entity for groups of files that are generated together, and perhaps its name should be put here.

hmm, the idea was, that one generator can create many files, so the files are below the generators in the hierarchy. Files belong to a generator in that they need to be generated by one, not sure where else in the option tree they could live

Not sure.

This seems to be a much more idiomatic way of writing it. That piece of code had confused the hell out of me, but maybe I'm just used to different things.

Only changing it to default = file.name would work to the same effect, but I still find it unusual.

uber nit: missing period at the end of this sentence?

With this API, is there any guarantee about the order the user is prompted? I guess I'd expect iteration over attrsets to be deterministic in nix, but would it be preferable to have an API that gives the developer control over the order of the prompts? Ideas:

• change this to an array (perhaps not module option merge friendly)
• add a z-index sort of number
• allowing prompts to depend on other prompts

attrValues always generates the prompts in lexicographical order. so it should be the same every time (at least for implementation that take this from bash). Prompts just take an input and output a string. so there is no real need to have a fixed order?

Can we add a value option or something similar to each prompt? I would like to be able to pre-define the value of the prompt to ensure that a user can always opt-in to a purely declarative configuration.

Btw, I currently don't understand when an author of a nixos module would decide to utilize a prompt instead of a generated random value. If I am writing a generator for my personal configuration, I can understand that I might want to use prompt for certain things - but when would I want to use a prompt in an upstream module and force interactivity on every user?

I'm just a little worried right now that introducing prompts without a way to declaratively set their value will sooner or later result in some generators being added to nixpkgs which depend on some form of interactivity. Therefore I'd like to have a way to specify the value of a prompt ahead of time, so the generation script can then just copy the specified file to prompts/$name instead of asking the user a question.

we mainly use prompts for API keys, which we usually don't want to leak into the config. I guess we can think about pre declaring things, this would also make the testing code a bit easier which I worked on last time I touched that code. But this increases the surface every backend implementation has to take care of.

nit (feel free to not make any change here): This option smells a bit off to me: nix makes it so easy for people to build their own scripts/binaries with all their own dependencies: why are we doing this for them?

Is this just to make things a little more ergonomic for users? For example, this might save people from an extra layer or two of indentation that would be introduced by using a trivial builder (such as writerShellApplication).

This seems like an odd default. Should we set it to null to help people remember to actually define a script?

scripts are optional in our outside implementation, because in the case that there is prompt but no script, the prompt automatically gets persisted as a secret file (which can be referenced again) I removed this shortcut for now, to have less complexity. But we could add it back at a later point maybe

This placed seemed pretty ordered until this line got added. Or is all-tests.nix also just kindasorted like module-list.nix :)