Skip to content

Linkml conversion tooling#387

Draft
yarikoptic wants to merge 23 commits intomasterfrom
linkml-conversion
Draft

Linkml conversion tooling#387
yarikoptic wants to merge 23 commits intomasterfrom
linkml-conversion

Conversation

@yarikoptic
Copy link
Copy Markdown
Member

@yarikoptic yarikoptic commented Mar 20, 2026

This is an extract with amends from

which (branch linkml-auto-converted) would keep merging this branch into itself while reflecting on changes in the branch which could be rebased or gain merges from the master, and also can accumulate or drop "patch branches" from within its script defining what to patch with.

This way linkml-auto-converted would represent reflection of current state of conversion

TODO/PLAN

  • Establish branch linkml-auto-converted -- that one in WiP: Branch with auto converted linkml model #381
  • Made ‘hatch’ script (you could add pydantic2linkml as dependency there) to convert orig_models.py into dandischema/models.yaml : hatch ... TODO
  • Translated the original models.py into dandischema/models.yaml and overlaid with an [dandischema/models_overlay.yaml] overlay file.
  • script tools/linkml_conversion to convert into ‘linkml-auto-converted’
  • define model_instances.yaml (or alike) which would define pre-populated records such as standards (bids, nwb, ...). aim for potentially multiple classes there.
  • add a github workflow here which would react to changes into 'master' and this branch and with manual dispatch, which would first merge master into this branch, then run the script, and push results to linkml-auto-converted branch. This way we would always have 'up to date' and automatically updated state of that branch.
  • address "notes" about failed conversions one way (changing current dandi-schema pydantic model) or another (pydantic2linkml) or !
    • we can add a custom script to "enhance" auto generate linkml model to address any changes needed programmatically!
    • we can have a branch (or just a .patch file) with changes to perform on top of converted linkml
  • ...
  • There you produce pydantic model out of this patched model sufficient (although potentially more relaxed) to replace current pydantic model.

candleindark and others added 17 commits March 13, 2026 17:01
 Specify Hatch-managed env for auto converting
 `dandischema.models` to LinkML schema and
 back to Pydantic models
Provide script to translate `dandischema.models`
in to a LinkML schema and overly it with
definition provided by an overlay file.
Provide script to translate `dandischema/models.yaml`
back to Pydantic models and store them in
`dandischema/models.py`
The previous BRE pattern used `\+` (GNU sed extension) which silently
fails
on macOS BSD sed. Switch to `-E` (extended regex) with POSIX character
class
`[^[:space:]]` instead of `\S` (also unsupported by BSD sed), making the
normalization work on both macOS and Linux.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Expand comment for linkml-auto-converted hatch env with usage instructions

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
There is no prefix defined as `dandi_default`.
The intended default prefix is `dandi`
…ed and some symbols from _orig for now

we do it so it does not overlay models.py since then git
is unable to track renames
we had to maintain original filename for models.py to apply patches
easily
# Poor man patch queue implementation
# Edit this list if you want to merge or drop PRs branches to be patched with.
# Order matters
branches_to_merge=( remove-discriminated-unions )
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that is where we define branches from PRs to merge!

@codecov
Copy link
Copy Markdown

codecov bot commented Mar 20, 2026

Codecov Report

❌ Patch coverage is 0% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 97.83%. Comparing base (4b89e4f) to head (e2d124c).
⚠️ Report is 2 commits behind head on master.

Files with missing lines Patch % Lines
dandischema/models_importstab.py 0.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #387      +/-   ##
==========================================
- Coverage   97.92%   97.83%   -0.09%     
==========================================
  Files          18       19       +1     
  Lines        2405     2407       +2     
==========================================
  Hits         2355     2355              
- Misses         50       52       +2     
Flag Coverage Δ
unittests 97.83% <0.00%> (-0.09%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants