From ace8079b72d7d363876457010c420bb5878cd196 Mon Sep 17 00:00:00 2001 From: jaimergp Date: Thu, 28 Aug 2025 16:44:47 +0200 Subject: [PATCH 1/8] Improve "How to teach this" section --- peps/pep-9999.rst | 90 ++++++++++++++--------------------------------- 1 file changed, 26 insertions(+), 64 deletions(-) diff --git a/peps/pep-9999.rst b/peps/pep-9999.rst index 2d3dd4ed215..fc89073984d 100644 --- a/peps/pep-9999.rst +++ b/peps/pep-9999.rst @@ -995,76 +995,19 @@ of the impact of the operation should be provided. How to Teach This ================= -There are at least four audiences that need to learn a workflow here. +There are at least three audiences that may need to get familiar with the contents of this PEP: -1. Python package maintainers wishing to express an external dependency. +1. Central registry maintainers, who are responsible for curating the list of well-known DepURLs + and mapped ecosystems. 2. Packaging ecosystem maintainers, who are responsible for keeping the - mapping for their ecosystem current. -3. Core registry maintainers, who are responsible for curating the central - repository of DepURLs and their descriptions. -4. End users of packages that have external dependency metadata. - -Python package maintainer usage -------------------------------- - -A package maintainer's responsibility is to decide the DepURL that best -represents the external dependency that their package needs. Their task -consists of: - -1. Understanding the nature of their dependency. Do they only need runtime - libraries, or do they need development packages for build-time concerns? - This understanding feeds into PEP 725, which specifies the expression of - these dependencies in metadata. -2. Looking up the DepURL. This can either mean knowing the name of the package - in their package ecosystem, and then inverse-mapping that to the DepURL, or - it can mean looking up the DepURL directly. -3. When a package maintainer does not find an appropriate mapping, they should look - for a fitting DepURL. It can be the case that although a DepURL is registered, not every - package ecosystem has a corresponding mapping. If no appropriate DepURL exists, - the package maintainer may consider submitting a new DepURL to the central registry. - -A prototype interactive mappings browser that showcases this workflow is available at -`external-metadata-mappings.streamlit.app `__. - -An overall workflow diagram might look like this: - -.. mermaid:: - - flowchart TD - A[Python package author with new external dependency] --> |Looks in| B(DepURL/description collection) - B --> | Find DepURL OK | E(Add DepURL to pyproject.toml) - A --> | Looks in | C(Ecosystem mapping file) - C --> | Finds familiar ecosystem package name | D(Inverse map ecosystem package name to DepURL) - D --> | Mapping exists | E - B --> | DepURL not found | F(Submit identifier proposal to DepURL/description collection) - F --> | Accepted | G(Mapping maintainers notified of missing DepURL mappings) - D --> | Mapping missing. User looks in DepURL collection. | B - B --> | Was mapping missing? | H(User may contribute entry to mapping) - -Package ecosystem maintainers usage ------------------------------------ - -Any packages that express a DepURL that does not have a mapping in a given package -ecosystem might not be able to provide tailored error messages and other UX affordances for end users. -It is thus recommended that each package ecosystem maintain their mappings. Key to this will -be automation. Some ideas for opt-in automation are: - -- Alert mapping maintainers whenever a new DepURL is added to the registry (maybe noisy). -- Provide tools that allow maintainers to diff their mappings to the registry contents to - quickly identify missing entries. -- Provide automated tooling that submits PRs to known mapping locations, such that maintainers - need only fill in the ecosystem package name. -- Provide status for each DepURL, to readily identify which DepURLs need attention. - -This maintenance is likely to involve a lot of work to establish the initial mapping, but ideally become small -on an ongoing basis. - + mapping for their ecosystem up-to-date. +3. End users of packages that have external dependency metadata. Central DepURL registry maintainers ----------------------------------- Central DepURL registry maintainers curate the collection of -DepURLs. These contributors need to be able to refer to clearly +DepURLs and the known ecosystems. These contributors need to be able to refer to clearly defined rules for when a new DepURL can be defined. It is undesirable to be loose with canonical DepURL definitions, because each definition implies maintenance in the mappings in many other places. @@ -1080,13 +1023,32 @@ Having client-side validation when the Python project is being packaged and/or u to PyPI may help keep the maintenance efforts contained, since end-users can be pointed to the recommended identifiers. +Package ecosystem maintainers usage +----------------------------------- + +Any packages that express a DepURL that does not have a mapping in a given +package ecosystem will not be able to provide tailored error messages and other +UX affordances for end users. It is thus recommended that each package +ecosystem maintain their mappings up-to-date with the central registry. Key to +this will be automation. Some ideas for opt-in automation are: + +- Alert mapping maintainers whenever a new DepURL is added to the registry (maybe noisy if too frequent). +- Provide tools that allow maintainers to diff their mappings to the registry contents to + quickly identify missing entries. +- Provide automated tooling that submits PRs to known mapping locations, such that maintainers + need only fill in the ecosystem package name. + +This maintenance is likely to involve a lot of work to establish the initial mapping, but ideally become small +on an ongoing basis. + + End user package consumers -------------------------- There will be no change in user experience by default. End users do not need to know about this mechanism unless they opt in, which they may want to do to, for example, reduce their bandwidth and disk space usage. This is particularly true if the user only relies on wheels, -since the only impact will be driven by external runtime dependencies. +since the only impact will be driven by external runtime dependencies (expected to be rare). If they do opt-in, in an ideal case these package install commands can be done transparently, and the user experience remains unchanged. There are several foreseeable issues that will arise, From e1b3595096c1d68dda40ed0fd311d3fdabb601a6 Mon Sep 17 00:00:00 2001 From: jaimergp Date: Thu, 28 Aug 2025 16:56:07 +0200 Subject: [PATCH 2/8] Readd Package maintainers --- peps/pep-9999.rst | 30 ++++++++++++++++++++++++------ 1 file changed, 24 insertions(+), 6 deletions(-) diff --git a/peps/pep-9999.rst b/peps/pep-9999.rst index fc89073984d..55ca2bf68b8 100644 --- a/peps/pep-9999.rst +++ b/peps/pep-9999.rst @@ -1001,7 +1001,8 @@ There are at least three audiences that may need to get familiar with the conten and mapped ecosystems. 2. Packaging ecosystem maintainers, who are responsible for keeping the mapping for their ecosystem up-to-date. -3. End users of packages that have external dependency metadata. +3. Maintainers of Python projects that require external dependencies. +4. End users of packages that have external dependency metadata. Central DepURL registry maintainers ----------------------------------- @@ -1041,14 +1042,30 @@ this will be automation. Some ideas for opt-in automation are: This maintenance is likely to involve a lot of work to establish the initial mapping, but ideally become small on an ongoing basis. +Maintainers of Python projects +------------------------------ + +A package maintainer's responsibility is to decide the DepURL that best +represents the external dependency that their package needs. Their task +consists of: + +1. Understanding the nature of their dependency. Do they only need runtime + libraries, or do they need development packages for build-time concerns? + They should refer to :doc:`pep-0725` for more details. +2. Looking up the DepURL in the central registry, or via a package name in + suitable ecosystem-specific mapping. A prototype interactive mappings browser + that may come helpful in these cases can be located at + `external-metadata-mappings.streamlit.app `__. +3. If no DepURL is available, consider submitting a request in the central registry. + End user package consumers -------------------------- -There will be no change in user experience by default. End users do not need to know about -this mechanism unless they opt in, which they may want to do to, for example, reduce their -bandwidth and disk space usage. This is particularly true if the user only relies on wheels, -since the only impact will be driven by external runtime dependencies (expected to be rare). +There will be no change in user experience by default. This is particularly +true if the user only relies on wheels, since the only impact will be driven by +external runtime dependencies (expected to be rare), and even in those cases +they need to opt-in by installing a compatible tool. If they do opt-in, in an ideal case these package install commands can be done transparently, and the user experience remains unchanged. There are several foreseeable issues that will arise, @@ -1059,7 +1076,8 @@ though: tool (e.g. system Python users). These issues might impact the user experience with untailored error messages for the chosen -ecosystem, permission errors reports, and so on. +ecosystem, permission errors reports, and so on. They may choose to help remediate this by +reporting missing entries in their target ecosystem, or even creating a new one if absent. Reference Implementation ======================== From 787f1e0e46f7324c5c4ea16f8ff2f6defca86d98 Mon Sep 17 00:00:00 2001 From: jaimergp Date: Thu, 28 Aug 2025 16:56:36 +0200 Subject: [PATCH 3/8] four --- peps/pep-9999.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/peps/pep-9999.rst b/peps/pep-9999.rst index 55ca2bf68b8..d56efb5b1ca 100644 --- a/peps/pep-9999.rst +++ b/peps/pep-9999.rst @@ -995,7 +995,7 @@ of the impact of the operation should be provided. How to Teach This ================= -There are at least three audiences that may need to get familiar with the contents of this PEP: +There are at least four audiences that may need to get familiar with the contents of this PEP: 1. Central registry maintainers, who are responsible for curating the list of well-known DepURLs and mapped ecosystems. From 6428e1c5c2f67d1f16efd0bf087a47e8e6d2a21b Mon Sep 17 00:00:00 2001 From: jaimergp Date: Thu, 28 Aug 2025 17:02:18 +0200 Subject: [PATCH 4/8] trim some sections --- peps/pep-9999.rst | 69 +++++++++++++++++++---------------------------- 1 file changed, 27 insertions(+), 42 deletions(-) diff --git a/peps/pep-9999.rst b/peps/pep-9999.rst index d56efb5b1ca..c52eb9009d0 100644 --- a/peps/pep-9999.rst +++ b/peps/pep-9999.rst @@ -997,8 +997,8 @@ How to Teach This There are at least four audiences that may need to get familiar with the contents of this PEP: -1. Central registry maintainers, who are responsible for curating the list of well-known DepURLs - and mapped ecosystems. +1. Central registry maintainers, who are responsible for curating the list of + well-known DepURLs and mapped ecosystems. 2. Packaging ecosystem maintainers, who are responsible for keeping the mapping for their ecosystem up-to-date. 3. Maintainers of Python projects that require external dependencies. @@ -1007,57 +1007,42 @@ There are at least four audiences that may need to get familiar with the content Central DepURL registry maintainers ----------------------------------- -Central DepURL registry maintainers curate the collection of -DepURLs and the known ecosystems. These contributors need to be able to refer to clearly -defined rules for when a new DepURL can be defined. It is -undesirable to be loose with canonical DepURL definitions, because -each definition implies maintenance in the mappings in many other places. - -The ``provides`` key mechanism offer ways to maintain aliases, so hopefully a -compromise of flexibility and strictness can be found easily. Particular attention -must be put to deciding which of the aliases will be the canonical form, though, -especially when it comes to dependencies where a number of synonyms are commonly -used. This does not apply to ``dep:virtual/*`` identifiers, where a single canonical -form is proposed and no additional aliases are allowed. - -Having client-side validation when the Python project is being packaged and/or uploaded -to PyPI may help keep the maintenance efforts contained, since end-users can be pointed -to the recommended identifiers. +Central DepURL registry maintainers curate the collection of DepURLs and the +known ecosystems. These contributors need to be able to refer to clearly +defined rules for when a new DepURL can be defined. It is undesirable to be +loose with canonical DepURL definitions, because each definition added implies +maintenance in the mappings in the target ecosystems. Package ecosystem maintainers usage ----------------------------------- -Any packages that express a DepURL that does not have a mapping in a given -package ecosystem will not be able to provide tailored error messages and other -UX affordances for end users. It is thus recommended that each package -ecosystem maintain their mappings up-to-date with the central registry. Key to -this will be automation. Some ideas for opt-in automation are: +Missing mapping entries will result in the absence tailored error messages and +other UX affordances for end users of the impacted ecosystems. It is thus +recommended that each package ecosystem maintain their mappings up-to-date with +the central registry. Key to this will be automation. Some ideas for opt-in +automation are: -- Alert mapping maintainers whenever a new DepURL is added to the registry (maybe noisy if too frequent). -- Provide tools that allow maintainers to diff their mappings to the registry contents to - quickly identify missing entries. -- Provide automated tooling that submits PRs to known mapping locations, such that maintainers - need only fill in the ecosystem package name. +- Alert mapping maintainers whenever a new DepURL is added to the registry + (maybe noisy if too frequent). +- Provide tools that allow maintainers to diff their mappings to the registry + contents to quickly identify missing entries. +- Provide automated tooling that submits PRs to known mapping locations, such + that maintainers need only fill in the ecosystem package name. -This maintenance is likely to involve a lot of work to establish the initial mapping, but ideally become small -on an ongoing basis. +This maintenance is likely to involve a lot of work to establish the initial +mapping, but ideally become small on an ongoing basis. Maintainers of Python projects ------------------------------ A package maintainer's responsibility is to decide the DepURL that best -represents the external dependency that their package needs. Their task -consists of: - -1. Understanding the nature of their dependency. Do they only need runtime - libraries, or do they need development packages for build-time concerns? - They should refer to :doc:`pep-0725` for more details. -2. Looking up the DepURL in the central registry, or via a package name in - suitable ecosystem-specific mapping. A prototype interactive mappings browser - that may come helpful in these cases can be located at - `external-metadata-mappings.streamlit.app `__. -3. If no DepURL is available, consider submitting a request in the central registry. +represents the external dependency that their package needs. This is covered +in :doc:`pep-0725`; the interactive mappings browser demo located at +`external-metadata-mappings.streamlit.app `__ +may come handy. +If no suitable DepURL is available for a given dependency, maintainers may +consider submitting a request in the central registry. End user package consumers -------------------------- @@ -1067,7 +1052,7 @@ true if the user only relies on wheels, since the only impact will be driven by external runtime dependencies (expected to be rare), and even in those cases they need to opt-in by installing a compatible tool. -If they do opt-in, in an ideal case these package install commands can be done transparently, +If they do opt-in, in an ideal case these external commands can be executed transparently, and the user experience remains unchanged. There are several foreseeable issues that will arise, though: From 57eff663eb52dfe4604d603e3e0896e342b2a66a Mon Sep 17 00:00:00 2001 From: jaimergp Date: Thu, 28 Aug 2025 17:19:34 +0200 Subject: [PATCH 5/8] simplify ecosystem maintainers list --- peps/pep-9999.rst | 12 +++--------- 1 file changed, 3 insertions(+), 9 deletions(-) diff --git a/peps/pep-9999.rst b/peps/pep-9999.rst index c52eb9009d0..b44948f4c11 100644 --- a/peps/pep-9999.rst +++ b/peps/pep-9999.rst @@ -1019,15 +1019,9 @@ Package ecosystem maintainers usage Missing mapping entries will result in the absence tailored error messages and other UX affordances for end users of the impacted ecosystems. It is thus recommended that each package ecosystem maintain their mappings up-to-date with -the central registry. Key to this will be automation. Some ideas for opt-in -automation are: - -- Alert mapping maintainers whenever a new DepURL is added to the registry - (maybe noisy if too frequent). -- Provide tools that allow maintainers to diff their mappings to the registry - contents to quickly identify missing entries. -- Provide automated tooling that submits PRs to known mapping locations, such - that maintainers need only fill in the ecosystem package name. +the central registry. Key to this will be automation, like linting scripts +(see example at `external-metadata-mappings `__), +or periodic notifications via issues or draft submissions. This maintenance is likely to involve a lot of work to establish the initial mapping, but ideally become small on an ongoing basis. From 45e79ff40ccab3163b3a564044a75e460fe21d5f Mon Sep 17 00:00:00 2001 From: jaimergp Date: Thu, 28 Aug 2025 17:25:09 +0200 Subject: [PATCH 6/8] trim potential problems for end users if mappings are incomplete --- peps/pep-9999.rst | 16 +++++----------- 1 file changed, 5 insertions(+), 11 deletions(-) diff --git a/peps/pep-9999.rst b/peps/pep-9999.rst index b44948f4c11..e257013fc1d 100644 --- a/peps/pep-9999.rst +++ b/peps/pep-9999.rst @@ -1046,17 +1046,11 @@ true if the user only relies on wheels, since the only impact will be driven by external runtime dependencies (expected to be rare), and even in those cases they need to opt-in by installing a compatible tool. -If they do opt-in, in an ideal case these external commands can be executed transparently, -and the user experience remains unchanged. There are several foreseeable issues that will arise, -though: - -* A mapping does not exist for the user's desired package ecosystem. -* A user does not have permissions to run the install commands provided by our - tool (e.g. system Python users). - -These issues might impact the user experience with untailored error messages for the chosen -ecosystem, permission errors reports, and so on. They may choose to help remediate this by -reporting missing entries in their target ecosystem, or even creating a new one if absent. +Users that do opt-in may find missing entries in for their target ecosystems, and +in those cases they may choose to collaborate by reporting the missing entries, +submitting a fix to the affected mapping or, if totally absent, even deciding +to maintain a new one on their own. To that end, they should get familiar with +the responsibilties of mapping maintainers (discussed above). Reference Implementation ======================== From 60905bddac28b8d9551d75176ec7309dcc0b2545 Mon Sep 17 00:00:00 2001 From: jaimergp Date: Thu, 28 Aug 2025 18:44:59 +0200 Subject: [PATCH 7/8] Apply suggestions from code review MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Michał Górny --- peps/pep-9999.rst | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/peps/pep-9999.rst b/peps/pep-9999.rst index e257013fc1d..4052bc25360 100644 --- a/peps/pep-9999.rst +++ b/peps/pep-9999.rst @@ -1010,21 +1010,20 @@ Central DepURL registry maintainers Central DepURL registry maintainers curate the collection of DepURLs and the known ecosystems. These contributors need to be able to refer to clearly defined rules for when a new DepURL can be defined. It is undesirable to be -loose with canonical DepURL definitions, because each definition added implies -maintenance in the mappings in the target ecosystems. +loose with canonical DepURL definitions, because each definition added increases +maintenance effort in the mappings in the target ecosystems. Package ecosystem maintainers usage ----------------------------------- Missing mapping entries will result in the absence tailored error messages and other UX affordances for end users of the impacted ecosystems. It is thus -recommended that each package ecosystem maintain their mappings up-to-date with -the central registry. Key to this will be automation, like linting scripts +recommended that each package ecosystem keeps their mappings up-to-date with +the central registry. The key to this will be automation, like linting scripts (see example at `external-metadata-mappings `__), or periodic notifications via issues or draft submissions. -This maintenance is likely to involve a lot of work to establish the initial -mapping, but ideally become small on an ongoing basis. +Establishing the initial mapping is likely to involve a lot of work, but ideally the maintenance on an ongoing basis effort should require smaller effort. Maintainers of Python projects ------------------------------ @@ -1041,7 +1040,7 @@ consider submitting a request in the central registry. End user package consumers -------------------------- -There will be no change in user experience by default. This is particularly +There will be no change in the user experience by default. This is particularly true if the user only relies on wheels, since the only impact will be driven by external runtime dependencies (expected to be rare), and even in those cases they need to opt-in by installing a compatible tool. From c15678be31f6e4049b853f15dee285a1ff210347 Mon Sep 17 00:00:00 2001 From: jaimergp Date: Mon, 1 Sep 2025 12:19:41 +0200 Subject: [PATCH 8/8] Mention documentation as the main source for learning material --- peps/pep-9999.rst | 22 ++++++++++++++++++---- 1 file changed, 18 insertions(+), 4 deletions(-) diff --git a/peps/pep-9999.rst b/peps/pep-9999.rst index e257013fc1d..9550559f04d 100644 --- a/peps/pep-9999.rst +++ b/peps/pep-9999.rst @@ -1013,6 +1013,10 @@ defined rules for when a new DepURL can be defined. It is undesirable to be loose with canonical DepURL definitions, because each definition added implies maintenance in the mappings in the target ecosystems. +The central registry maintainers should agree on the ground rules and write them +down as part of the repository documentation, perhaps supported by additional +affordances like issue and pull request templates, or linting tools. + Package ecosystem maintainers usage ----------------------------------- @@ -1026,6 +1030,10 @@ or periodic notifications via issues or draft submissions. This maintenance is likely to involve a lot of work to establish the initial mapping, but ideally become small on an ongoing basis. +As best practices are discovered and agreed on, they should get documented +in the central registry repository as learning materials for the mapping +maintainers. + Maintainers of Python projects ------------------------------ @@ -1033,10 +1041,12 @@ A package maintainer's responsibility is to decide the DepURL that best represents the external dependency that their package needs. This is covered in :doc:`pep-0725`; the interactive mappings browser demo located at `external-metadata-mappings.streamlit.app `__ -may come handy. +may come handy. The central registry documentation may include examples and +frequently asked questions to guide newcomers with their decisions. If no suitable DepURL is available for a given dependency, maintainers may -consider submitting a request in the central registry. +consider submitting a request in the central registry. Instructions on how to do +this should be provided as part of the central registry documentation. End user package consumers -------------------------- @@ -1046,8 +1056,12 @@ true if the user only relies on wheels, since the only impact will be driven by external runtime dependencies (expected to be rare), and even in those cases they need to opt-in by installing a compatible tool. -Users that do opt-in may find missing entries in for their target ecosystems, and -in those cases they may choose to collaborate by reporting the missing entries, +Users that do opt-in may find missing entries in for their target ecosystems, for +which they should obtain informative error messages that point to the relevant +documentation sections. This will allow them to get acquainted with the nature +of the issue and its potential solutions. + +We hope that this results in a subset of them reporting the missing entries, submitting a fix to the affected mapping or, if totally absent, even deciding to maintain a new one on their own. To that end, they should get familiar with the responsibilties of mapping maintainers (discussed above).