Replies: 6 comments 4 replies
-
|
I'd appreciate feedback from
and whoever is interested in that topic. The proposal changes a lot and your feedback is very valuable to me. |
Beta Was this translation helpful? Give feedback.
-
|
Could we take #1432 in consideration, too? |
Beta Was this translation helpful? Give feedback.
-
|
I'm not sure about the use case described in #1432.
Aren't namespaces fixing this already? Please clarify if I miss your point(s). |
Beta Was this translation helpful? Give feedback.
-
|
One thing that comes to mind: The concept outlined https://github.com/useblocks/bazel-drives-sphinx basically dissolves Sphinx projects into collection of files. In that realm it actually makes very much sense to assign namespaces to files or whole folders. |
Beta Was this translation helpful? Give feedback.
-
|
Is that what you want, multiple namespaces in a single project? |
Beta Was this translation helpful? Give feedback.
-
|
Proposal from the
|
Beta Was this translation helpful? Give feedback.
-
|
Beta Was this translation helpful? Give feedback.
-
|
Really great write up. separator
For me hash
I hash in the rst code, which can stay stable even if the need got changed, is just a label and can be used like the version field, or not? As compromise, special internal options could be introduced, which indicated that the given value is just for information reasons and is changed/reset during build. .. req:: Burger req
:id: REQ_BURGER
:_hash: deadbeefneed namespace originIt would be great if the namespace could also have an url, where to find the source/generated docs of the urn. Something like: needs_namespaces = {
"sn": {
"urn": "urn:useblocks:sphinx-needs",
"docs": "http://company.com/my_docs/permalink.html?q={id}",
"code_view": "http:/github.com/company/my_docs/view/{docname}.html#{id},
"code_edit": "http:/github.com/company/my_docs/edit/{docname}.html#{id}
}
}This would allows builders to create URLs for needs referenced by urn. |
Beta Was this translation helpful? Give feedback.
-
|
@ubmarco +1 for using an existing standard, i.e. https://en.wikipedia.org/wiki/Uniform_Resource_Identifier,
|
Beta Was this translation helpful? Give feedback.
-
|
I love the hashing concept, and have previously implemented something like this in a different framework. I definitely have an invested interest in this feature. Another idea beyond Dan's is to use https://codelinks.useblocks.com/components/write.html This would allow for different use cases to have a separation of concerns for the metadata. Another use case for using both hash and version would be that the automatic hashing concept helps hint to the developers of the changed content that the version needs to be bumped (of course they have to choose whether it is a "typo", "backwards compat", "breaking change") e.g. build fails because hash stored in the rst (either inline or a needextend) differs from what's computed at runtime I feel like folks of traditional ALM tools are used to talking in terms of different "versions" of content rather than "hashes". I would probably want to use "versions" for my content to define/link against, but the safety net of hashing would be great. I am also curious what kind of additional tooling support would be possible once this feature is added in tools such as ubtrace to view content based upon content versioning over time or hash versioning over time |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
This concept takes the existing GitHub discussions:
forward and proposes an implementation.
Other use cases of Sphinx-Needs are also considered, such as:
Current state
Sphinx-Needs currently employs the following link semantics:
as well
that imports the needs
(network validation)
Currently, the serialized and deserialized forms of links are identical, i.e. the link specified
in RST is the same as the link stored in the internal data structures and in needs.json files.
Currently, each link is either a simple need ID like in
REQ_BURGERor the need ID combined with aneed part ID,
separated by a dot
.like inREQ_BURGER.tastyandREQ_BURGER.hot.Goals & Requirements
Going forward, the link semantics shall be extended to support edge properties that further
constrain the target need, which may include:
The concept should also consider the following requirements:
Ideas like hashing data requires good tool support. If that is not available, humans
will not stick to the concept and all the good ideas are lost.
if the target need does not have the right 'link quality', i.e. has a more recent version than
specified in the outgoing link. Users should be put in a position to
multi-repository setup.
use the data without having to parse the RST files.
an outgoing link is local to the project or not. That makes it possible for needs.json inventory builds
to suppress only dead external links warnings, which are expected to be resolved at a later stage.
See also discussions
#1166 How to get bidirectional link between two documents from different repos
and
#1220 Sphinx-needs HTML for modular sphinx projects with bi-directional links
.
Proposed solution
The proposed solution is a combination of a namespace
concept that follows the RFC 8141 for
Uniform Resource Names (URNs) and a serialization format for edge properties that is easy to write
for developers. Also an extension mechanisms is proposed to allow for custom edge properties.
Links will be serialized in the following format:
Quickstart
Some quick examples:
With a mapping configuration of
links to external needs can be simplified to:
Namespace concept
Namespaces should follow the standard RFC 8141 for
Uniform Resource Names (URNs). The general structure of a URN is:
where
<NID>is the namespace identifier and<NSS>is the namespace-specific string.In
useblocksis theNIDandsphinx-needsis theNSSpart of the URN.The
NSSpart can also become longer, e.g. to model company departments ortechnical subsystems:
Keep in mind the
:is not meant to be a hierachical separator within theNSSpart.It just separates the
urnfrom theNID, theNIDfrom theNSSand theNSSfrom the need ID.Any
NIDidentifier can however decide to impose such a meaning.The namespace
NSSis followed by the need ID and must be separated from itwith a colon
:.The namespace of the local project is configured in the Sphinx configuration file
conf.pyor in theubproject.tomlfile usingAll local project needs as well as local links automatically get this namespace.
It does not have to be specified in RST and it is automatically applied when exporting data.
Referencing needs from external projects require the fully specified need ID including
a namespace prefix. Alternatively an abbreviation key can be mapped to the target
namespace:
Versioning concept
Needs can be versioned manually within the documentation by defining a
version on the need itself. The versioning is completely independent of any version control
system. Using it can lead to additional documentation maintenance efforts in exchange for
safer and more granular control of requirement updates and project updates.
Multiple versioning styles have been proposed in the past:
major: simple integer numbermajor.minor: major and minor integermajor.minor.patchlike in Semantic Versioninghashbased versioningI propose the
major.minor.patchidea as it allows for different levels of changes.You may increment the:
majorversion if you make incompatible need changesminorversion if you add requirement information in a compatible waypatchversion when you make backwards compatible changes, e.g. fix typosProjects may also decide to run a simplified versioning concept by only using
the
majorversion and just not updating it for compatible changes.The need version is defined in RST with a new core field
version:The version string is attached to the need link using the
@<version>syntax.Note
The version is an additional link property and can be seen like
a query parameter in a URL. It is not required to
resolve the need.
Hashing concept
In the above versioning concept, developers decide about version updates.
That leaves room for mistakes, so need items can change significantly without
the knowledge of their dependencies. A hash-based approach is introduced that can help in these
scenarios. Sphinx-Needs will pick a subset of the need fields and calculate a unique hash
which is then either stored in RST or held in memory. Example:
The details of the calculation, the algorithm and the hash management including tool support need to be
further specified. This section is about the serialization of the hash in need links.
The hash string is attached to the need link using the
#<hash>syntax.Note
The hash is an additional link property and can be seen like
a query parameter in a URL. It is not required to
resolve the need.
Allowed characters and escaping
The namespace start string
urn:is case insensitive.The namespace
NIDmust follow the regex[a-zA-Z][a-zA-Z0-9-]{0,31}as per RFC 8141.It is case insensitive.
The namespace
NSSis more open. In the implementation for Sphinx-Needsit allows for
[a-zA-Z][a-zA-Z0-9-,_:&~\/\[\]\(\)]*The following characters are reserved in link strings:
:may only appear betweenurnand the need ID; the last:identifies the start of the need ID.separates needs from need parts and separates the major/minor/patch semantic version numbers@indicates the start of the version field#indicates the start of the hash field\is used as an escape character to allow the above characters in need IDs and parts?indicates the start of a query component (future extension)=separates the query component name from its value;separates multiple query componentsDue to the extension of the link serialization options, some symbols
are now disallowed that were previously allowed in a need ID.
That might lead to adoption issues for some projects.
Therefore a backslash-based escape mechanism is added that removes the
special meaning of above characters. Example:
urn:useblocks:sphinx-needs:NEED_WITH_\@_AND_\:_COLONFurther considerations
File wide metadata
Sphinx supports a
file-wide metadata section
in RST files. That can be leveraged to
needs_namespaceconfig orCustom fields
The URN standard allows for custom query components.
This could be used to add custom edge properties to the link serialization.
For example, a custom field
customcould be added to the link serialization:For this to be supported, use cases should be collected and discussed first.
Deserialization
The deserialization of the link string is straightforward. The link string is split
by the reserved characters and the parts are assigned to the respective fields.
The deserialization process follows a left-to-right parsing approach:
urn:(case insensitive).If yes, find the last
:to separate namespace from need ID.Everything before the last
:is the namespace, everything after is the need ID with potential suffixes..to separate need ID from part ID. The first segment is the need ID, any additional segments form the part ID.@in the need ID segment. If found, everything after@(until#or?if present) is the version.#in the remaining string. If found, everything after#(until?if present) is the hash.?and parsekey=valuepairs separated by;.\@,\:, etc.) by removing the backslash and treating the following character literally.Parsing order: namespace → need_id → part → version → hash → query parameters
Example parsing:
urn:useblocks:sphinx-needs:REQ_BURGER.tasty@1.0#deadbeefurn:useblocks:sphinx-needsREQ_BURGERtasty1.0deadbeefError handling: Invalid characters, malformed namespaces, or parsing conflicts should generate appropriate warnings or errors.
The deserialized data should be stored in a structured format, both internally
and in needs.json for easy access to the individual components.
Allowed characters for need IDs and need parts
This is still an open topic. The current implementation allows for all printable characters
excluding the
.which separates need IDs from need parts.It is only constrained by the
needs_id_regexconfiguration option.Going foward it is helpful to avoid above described reserved characters so escaping
becomes less frequent.
Beta Was this translation helpful? Give feedback.
All reactions