feat: ds-dictionary redesign

[erikzwart]

Decide

• keep using current git repository for dictionaries or create ‘new’ repository on molgenis/dataschield?
• Update/change current dictionaries to new format or leave unchanged.
• keep using current git repository for dsUpload or create new repositry on molgenis/dataschield (dataschield cran?).
• Should ‘new’ dsUpload be compatible with old dictionary format.
• What should be the name of the new dsUpload or keep use of old name.
• dictionary file format csv or yml
• use regexp or javascript in dictionary file

Guidelines

• dictionary format is based on Opal template
• dictionary format is extended by validation rule column for variable and value
• dictionary documentation should be very clear on column format and options
• dictionary changes are made in the Variables.csv/yml and Categories.csv/yml for version control
• dictionary is defined as 1 file instead of multiple (non-rep, trimester, yearly)
• dictionary excel is created for the users for reference and sharing
• dictionary needs util script to create excel file
• dictionary needs util script to validate dictionary format
• dictionary needs util script to create automated changelogs
• dictionary/dsUpload needs util script to create valid emx2 dictionary
• dsupload user input should be either long or wide
• dsupload should be able to read maelstrom as dictionary input

TODO

• documentation
• setup dir structure
• add (opal) dictionary template
• dictionaries-source
• automate changelogs (diff dictionary versions)
• Opal and Armadillo compatible
• Validation rules in dictionaries-source
• Dictionary rules in dictionaries-source
• quality script(s) for dictionaries-source
• (optional) export/import emx2 data-catalogue script