Skip to content
/ rfcs Public
Draft
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
4801546
RFC 197: package set definitons initial commit
quantenzitrone Jan 24, 2026
9573e0f
Idea 1: easy transition advantage
quantenzitrone Jan 24, 2026
bb11e6a
collapsible idea sections test
quantenzitrone Jan 25, 2026
625449c
collapsible sections worked, so all of them now
quantenzitrone Jan 25, 2026
1d7fb7f
`tree` like output for visualization
quantenzitrone Jan 25, 2026
2c7fc9f
Update rfcs/0197-package-set-definitions.md
quantenzitrone Jan 25, 2026
b0c8c39
transform idea 1 to be more strict about the directory structure
quantenzitrone Jan 25, 2026
a2853b8
add python3Packages by-name to prior art
quantenzitrone Jan 25, 2026
84b5686
Idea3: attrpath in directory names drawback
quantenzitrone Jan 25, 2026
c592d41
Idea {2,3}: inaccessibility of sets like lixPackages
quantenzitrone Jan 25, 2026
ee5f96d
Idea {2,3}: package sets in pkgs/by-name is sketchy
quantenzitrone Jan 25, 2026
d47b2ad
more concrete numbers
quantenzitrone Jan 25, 2026
882c21a
Idea 1 only allows top-level package sets
quantenzitrone Jan 25, 2026
8f16dc0
another idea 3 drawback
quantenzitrone Jan 25, 2026
b26025e
scrap idea 3
quantenzitrone Jan 25, 2026
5c95506
improve goals
quantenzitrone Jan 25, 2026
2775b53
tooling has to be made to work for multiple directories
quantenzitrone Jan 25, 2026
53f107c
Update rfcs/0197-package-set-definitions.md
quantenzitrone Jan 25, 2026
3c4cbdd
Idea 1: redesign with auto called by-structure and without unsharded …
quantenzitrone Jan 25, 2026
2cf527e
Idea 2: drop, make idea 1 the default
quantenzitrone Jan 25, 2026
fa81fb3
minor fixes
quantenzitrone Jan 25, 2026
d0bb18b
move unresolved questions to proper location
quantenzitrone Jan 25, 2026
329891e
fix type
quantenzitrone Jan 25, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
254 changes: 254 additions & 0 deletions rfcs/0197-package-set-definitions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,254 @@
---
feature: nixpkgs package sets by-name
start-date: 2026-01-24
author: quantenzitrone
co-authors: (find a buddy later to help out with the RFC)
shepherd-team: (names, to be nominated and accepted by RFC steering committee)
shepherd-leader: (name to be appointed by RFC steering committee)
related-issues: https://github.com/NixOS/nixpkgs/pull/482538
---

# Summary
[summary]: #summary

Package sets like `fishPlugins` or `python3Packages` move to a `pkgs/by-name` like structure in
`pkgs/sets-by-name/<setname>`.

This doesn't apply to package sets that are auto-generated like `haskellPackages`.

# Motivation
[motivation]: #motivation

- get rid of the package categories as directories (decided in RFC 140 and 146)
- bring the benefits of by-name to package sets
- merge bot maintainer merging
- scaleability
- isolation
- vetting
- *your goal here*

This comment was marked as off-topic.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updates across package sets are very different. I don't think this should be a goal of this RFC, but instead something that can be developed separately as an updateScript that nixpkgs-update can pick up.


# Detailed design
[design]: #detailed-design

- Create a new directory under `pkgs/`, e.g. `pkgs/sets-by-name` that contains package sets.
- For example `pkgs/sets-by-name/fishPlugins`, `pkgs/sets-by-name/python3Packages`.
- Each package set is sharded like `pkgs/by-name`.
- The following additional have to exist for each package set.
- `functions.nix`: definitons of functions, like `buildFishPlugin`, `buildPythonPackage`.
- `overrides.nix`: overrides of packages, like `top-level/all-packages.nix` currently functions as
an overlay for `by-name` packages.
- This is something we try to keep empty. Most, maybe all, overrides can be inlined in the
package.
- `aliases.nix`: aliases for aliases in package sets behind `optionalAttrs config.allowAliases`
(like `top-level/aliases.nix`).
- All package sets with their sharded packages, overlayed with their functions, overrides and
aliases are automatically called by `top-level/package-sets-by-name.nix`.
- Versioned package sets like `python316Packages` are done in `all-packages.nix` by overriding the
default version package set.
- Versioned package sets without a default version will have to override the default version with
an error.
- e.g. `nextcloud*Packages` are in `sets-by-name/nextcloudPackages` and thus autocalled by
`top-level/package-sets-by-name.nix`, however we will have an alias in `top-level/aliases.nix`
that says
```nix
{
nextcloudPackages = throw "Please use nextcloudPackages for a specific nextcloud ersion e.g. nextcloud32Packages.";
}
```
```
pkgs
├── by-name
│ └── ...
├── sets-by-name
│ ├── fishPlugins
│ │ ├── as
│ │ │ ├── async-prompt
│ │ │ ... └── package.nix
│ │ ├── au
│ │ ...
│ │ ├── z_
│ │ │ └── z
│ │ │ └── package.nix
│ │ ├── aliases.nix
│ │ ├── functions.nix
│ │ └── overrides.nix
│ │
│ ├── python3Packages
│ │ ├── a2
│ │ │ └── a2wsgi
│ │ │ └── package.nix
│ │ ├── aa
│ │ ...
│ │ ├── zx
│ │ │ ├── zxcvbn
│ │ │ │ └── package.nix
│ │ │ ├── zxcvbn-rs-py
│ │ │ │ └── package.nix
│ │ │ └── zxing-cpp
│ │ │ └── package.nix
│ │ ├── aliases.nix
Comment on lines +83 to +90
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, a few files mixed within the shard list?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The alternative would be putting them in pkgs/top-level. It seems more sensible to have them at the root of the package set instead.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another alternative is to have one more level inside the package set before the shards (by-name subdirectory, I guess).

Which would also make more manageable packagesets that benefit from having a few more support files inside the package set but not as a packageset package.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That might require us to filter out that directory from the autocaller function, but it's a good idea as well.

Copy link
Author

@quantenzitrone quantenzitrone Jan 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think they meant

pkgs/sets
├── fishPlugins
│   ├── by-name
│   │   ├── as
│   │   │   ├── async-prompt
│   │   │   ... └── package.nix
│   │   ├── au
│   │   ...
│   │   └── z_
│   │       └── z
│   │           └── package.nix
│   └── top-level
│       ├── aliases.nix
│       ├── functions.nix
│       └── overrides.nix

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, it is a matter of taste if top-level is needed or the files would go into the root of the package set.

Copy link
Member

@Eveeifyeve Eveeifyeve Jan 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, it is a matter of taste if top-level is needed or the files would go into the root of the package set.
Exactly.

So I was thinking like this could happen:

pkgs/
├── fi/fishPlugins
│   ├── by-name
│   │   ├── as
│   │   │   ├── async-prompt
│   │   │   ... └── package.nix
│   │   ├── au
│   │   ...
│   │   └── z_
│   │       └── z
│   │           └── package.nix
└── aliases.nix
└── functions.nix
└── overrides.nix

By name is used in the case so that way you don't get confused easily with two by-names.
Another idea to improve this, is making the by-name in package sets to something like packages could work? So like:

pkgs/
├── fi/fishPlugins
│   ├── packages
│   │   ├── as
│   │   │   ├── async-prompt
│   │   │   ... └── package.nix
│   │   ├── au
│   │   ...
│   │   └── z_
│   │       └── z
│   │           └── package.nix
└── aliases.nix
└── functions.nix
└── overrides.nix

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you look at the POC implementation, you can also see that buildFishPlugin is in sets-by-name/fishPlugins/buildFishPlugin.nix.
For some package sets there might be larger amounts of functions so the package set folder could become crowded.
We could also just inline functions into functions.nix and require that no additional files are in the package set tree root.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand that. This mixes shards and non-shards under pkgs itself? And puts the package sets as normal packages? I am probably misreading the diagram.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand that. This mixes shards and non-shards under pkgs itself? And puts the package sets as normal packages? I am probably misreading the diagram.

Well the ideally in the packages it's another set then there would be default.nix which would hook it up.

I am probably misreading the diagram.

Probably 👀

│ │ ├── functions.nix
│ │ └── overrides.nix
│ ...
└── top-level
├── all-packages.nix <- calls all versioned package sets
├── by-name-overlay.nix <- used to autocall sharded packages (no change required)
...
└── package-sets-by-name.nix <- autocalls all sets by name
```
Proof-Of-Concept implementation in https://github.com/NixOS/nixpkgs/pull/482538
# Examples and Interactions
[examples-and-interactions]: #examples-and-interactions
TODO
# Drawbacks
[drawbacks]: #drawbacks
- only allows top level package sets
- *your drawback here*
# Alternatives
[alternatives]: #alternatives
<details>
<summary>
## Idea 2: nested by-name structure
</summary>
outdated Proof-Of-Concept implementation in https://github.com/NixOS/nixpkgs/pull/483432
### Detailed design
- Idea 1, but sets are in the existing `pkgs/by-name` structure instead of `pkgs/sets-by-name`, e.g.
`fishPlugins.puffer` would be in `by-name/fi/fishPlugins/pu/puffer`.
- Additionally a marker is required in order to distinguish package sets from simple packages,
such as using a `.packageset` file (example: `by-name/fi/fishPlugins/.packageset`).
If not the `package.nix` must exist and is called as a package.
```
pkgs
└── by-name
├── 0_
...
├── fi
│ ├── fiano
│ ├── fiche
│ ├── ...
│ ├── fishnet
│ ├── fishPlugins
│ │ ├── .packageset
│ │ ├── as
│ │ │ └── async-prompt
│ │ ├── au
│ │ ...
│ │ └── z_
│ │ └── z
│ ├── fishy
│ ...
...
```
### Advantages
- no new directory, just extend `pkgs/by-name`
- allows nested package sets
### Drawbacks
- `lixPackages` (behind all `lib*` packages) will not be accessible through GitHubs UI
- having package sets in `pkgs/by-name` may not fit the spirit of RFC 140
- *your drawback here*
</details>
<details>
<summary>
## Idea 3: package sets in `pkgs/by-name`
</summary>
Proof-Of-Concept implementation in https://github.com/NixOS/nixpkgs/pull/483128
### Detailed design
- Instead of `by-name/<shard>/<pname>` we have `by-name/<shard>/<attrpath>`, so e.g.
`fishPlugins.puffer` would go in `by-name/fi/fishPlugins.puffer`.
- The `top-level/by-name-overlay.nix` will call all folders in a `<shard>` that contain a dot as a
package set.
```
pkgs
└── by-name
├── 0_
...
├── fi
│ ├── fiano
│ ├── fiche
│ ├── ...
│ ├── fishnet
│ ├── fishPlugins.async-prompt
│ ├── fishPlugins.autopair
│ ├── fishPlugins.z
│ ├── fishy
│ ...
...
```
### Advantages
- no new directory, just extend `pkgs/by-name`
- allows nested package sets
### Drawbacks
- huge shards due to package sets
- currently only few shards like `li` are too large for GitHubs UI, but with this idea more shards
will be huge as well
- specifically 12 more shards `em`, `gn`, `ha`, `ho`, `oc`, `pe`, `py`, `rp`, `sb`,
`te`, `ty`, `vi` and `vi` (for `emacsPackages`, `gnomeExtensions`, `haskellPackages`,
`home-assistant-component-tests`, `ocamlPackages`, `pearlPackages`, `python3Packages`,
`rPackages`, `sbclPackages`, `texlivePackages`, `typstPackages` and `vimPlugins`) could become
inaccessible.
- some package sets like `lixPackages` (behind all `lib*` packages) will not be accessible through
GitHub UI
- having package sets in `pkgs/by-name` may not fit the spirit of RFC 140
- it's called pkgs/by-**name** and not pkgs/by-**attrpath**
- directory names as attrpaths is sketchy
- unresolved questions
### Unresolved Questions
- How do we handle functions like `fishPlugins.buildFishPlugin`?
Copy link

@SigmaSquadron SigmaSquadron Jan 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Answering because this applies to idea 1)

I never thought it made sense for functions and derivation builders to be inside package sets. I think our Go infrastructure has the right idea: move them to the top-level, and version them in the attribute name. fishPlugins.buildFishPlugin can just be moved to the top-level, while python313Packages.buildPythonApplication becomes buildPython313Application on the top-level. We can deal with this at stage 3, where we migrate everything over.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This proposal is awkward with Python where you have CPython and PyPy and then language version-dialects within them, and even worse for Common Lisp where you have same base language but multiple compatible implementations with their completely own versioning.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i also think keeping functions in package sets is fine

- How do we handle aliases?
- How do we handle versioned package sets?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Answering because this applies to idea 1)

I generally think it's a good idea to think of Nixpkgs as having three types of package infrastructure systems:

  1. Top-level packages, which nowadays go to by-name;
  2. Simple package sets, like fishPlugins, which are easier to implement in this new interface as we can simply call them as we do by-name packages and shove the resulting set into fishPlugins from the top-level.
  3. Versioned package sets, like python3Packages, which are by far the most complex. I think we should absolutely get the input of the primary maintainers of these sets here before we move forward. There are many concerns regarding updating versioned sets, and default aliases, as well as edge cases like python3.pkgs and python3Packages. In the end, having something that is no worse to maintain as it was before would be ideal.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Auto generated package sets like haskellPackages.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i want handle both 2 and 3 with this RFC

- How do we move large package sets?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Answering because this applies to idea 1)

Some of them are easy enough. If we resolve the aforementioned issues with the python edge cases, it should be a simple matter of moving and renaming files. OCaml should also be pretty simple.

The generated sets, like Haskell, Node, Lisp and R, will be impossible, though. Those packages don't actually exist as separate files in Nixpkgs, and are instead imported from some other upstream.

This is something that again, we'll have to get the input of package set maintainers on. Moving everything over will end up being a case-by-case basis.

</details>
# Prior art
[prior-art]: #prior-art
- `by-name` stucture for `python3Packages` https://github.com/NixOS/nixpkgs/pull/449896 https://github.com/NixOS/nixpkgs-vet/pull/180
- https://github.com/NixOS/nixpkgs/issues/482537
- https://github.com/NixOS/nixpkgs/issues/432625
- `tclPackages` has their own `by-name` structure https://github.com/NixOS/nixpkgs/pull/344716
- attempt to move `nushellPlugins` to `by-name` https://github.com/NixOS/nixpkgs/pull/482961
# Unresolved questions
[unresolved]: #unresolved-questions
- Does the handling of versioned package sets work like this?
- How do we move large package sets?
- *your question here*
# Future work
[future]: #future-work
What future work, if any, would be implied or impacted by this feature without being directly part of the work?