Add scripts to allow addons from personal repos to be synchronized with Crowdin

[nvdaes]

Blocked by #7

[seanbudd]

it's probably better to keep these rules than dropping to NVDA's standard

[nvdaes]

Purpose

Add-on authors may wish to help translators use Crowdin, the same framework where they translate NVDA. to translate messages and documentation for maintained add-ons:

Other details

• Pot file are created/updated, and uploaded to a Crowdin project.
• The readme.md file is converted to xliff and uploaded to a Crowdin project.
• Po and xfiles are translated.
• Translated files are downloaded and processed to be copied to locale/langCode/LC_MESSAGES/nvda.po, and doc/langCode/readme.md, in the addon folder.
  Authors need to store a Crowdin token with permissions to upload files to the Crowdin project as a repository secret.

Development approach

Use the Crowdin registration repo to add scripts usable by individual add-ons in personal repos.

[nvdaes]

I've tested that all check pass using this pyproject.toml file on this PR:

nvdaes/translateNvdaAddonsWithCrowdin#11

I use precommit, CodeQL and a workflow to check that all translatable messages have comments for translators.

I'll try to use the cache action to cache some add-on metadata like its id, and also hashfiles from l10nSources (taking the value of buildVars.py), and the hasf¡hfile of the readme.md, to determine if pot and xliff files should be updated.

[nvdaes]

Export translations to Crowdin running the workflow with update=False works properly:

https://github.com/nvdaes/translateNvdaAddonsWithCrowdin/actions/runs/19802210157

[nvdaes]

This time, updatexLiff is failing. Seems that adding blank lines to readme may cause problems:

https://github.com/nvdaes/translateNvdaAddonsWithCrowdin/actions/runs/19802391926/job/56731562709

[nvdaes]

If someone can help with this issue when update xliff, I'll be grateful.
I think that this is one of the bugest problems with xliff files. Sometimes sel lines are None and they don't have a strip method. I don't know if this should be also improved in NVDA
cc: @seanbudd

[seanbudd]

It might be easier to avoid xliff and just translate the markdown files directly. This won't support diffs very well but worth experimenting with

[nvdaes]

@seanbudd wrote:

It might be easier to avoid xliff and just translate the markdown files directly. This won't support diffs very well but worth experimenting with

OK.

[nvdaes]

@CyrilleB79, you were interested in this framework. If you want, feel free to see how the translateNvdaAddonsWithCrowdin.md can be translated in the project. Using xliff files is causing problems, as mentioned, and we are experimenting uploading md files instead.

[nvdaes]

I'm testing this locally with act and docker, with help of GitHub Copilot agent mode in VS Code. Maybe others want to test locally too, so Copilot has suggested a configuration for Docker to install gettext, and the configuration file is added to .gitignore by Copilot, reviewed by me. Changes in the workflow have been suggested by Copilot to install gettext, and I've asked it to add a dry-run to avoid upload and download from Crowsin, and also to avoid git push, not supported by act. I've also found out that 3.13 version is not recognized when testing locally, so we're using 3.13.11 for ubuntu.

[seanbudd]

Is it okay if we leave this as draft until #7 is resolved?

[nvdaes]

Sean wrote:

Is it okay if we leave this as draft until #7 is resolved?

Yes.

[seanbudd]

@nvdaes - now that nvaccess/nvdaL10n#1 is merged, can we use the l10nUtil exe from https://github.com/nvaccess/nvdaL10n/actions/runs/23521126188 here in #1 and close #7?

[nvdaes]

@seanbudd wrote:

can we use the l10nUtil exe from https://github.com/nvaccess/nvdaL10n/actions/runs/23521126188

I think it would be better to build the executable by checking out the repo, since the artifact may be deleted after the retention period. Let me know if you disagree.

[seanbudd]

I disagree - instead we could create a release and upload the artifact with every master push

[seanbudd]

To be clear - I think we should be building the release from nvaccess/nvdal10n. This repo and NVDA only needs the exe

[nvdaes]

I'm testing in the reportSymbols add-on, using the buildAddon workflow to build the pot file, though seems that Gettext can be installed on Windows too. But the buildAddon workflow is failing at least due to Pyright.
@CyrilleB79, if you can, please look at this

https://github.com/nvdaes/reportSymbols/actions/runs/23554914090/job/68578896225

[nvdaes]

Even removing code checks from build, the crowdin workflow is failing due to absence of modules like sha256. I'm thinking about uploading source files in all cases instead of checking if they should uploaded from scratch or updated, since the same command is used now. Previously the crowdinSync.py file was used for updates, but this is not the case now.

[nvdaes]

Seems that it's difficult to download the l10nUtil.exe file using gh (GitHub CLI), and it's not possible with curl.
Perhaps it's better to wait until you upload a release asset. Since the exe file is unzipped, this causes issues with GH cli.
I think that I won't use the build.yml workflow since it's too strict with code checks. I'll try to build the pot file with Ubuntu or Windows in the same workflow.

[seanbudd]

@nvdaes - can you create a workflow in nvaccess/nvdal10n to generate a release asset on push to main?

[nvdaes]

I think that it would be better to use the nvdaaddons/crowdinRegistration repo, or this template, to export translations periodically, and add-on authors would use their secrets.CROWDIN_TOKEN just to upload md and pot file to Crowdin.
I've tested exporting translations in the readFeeds add-on and this has taken more than 7 minutes. And if someone has many add-ons, it's time and api resources consuming.
Using an external repo just for this won't require extra work by admins of that repo like forking add-ons, since then the repo used to export translations can be checked out with GitHub Actions.
I've exported translations in the following workflow:

https://github.com/nvdaes/readFeeds/actions/runs/23920196250/job/69764310501

[nvdaes]

Anotheridea that I'm thinking about is to create an action (nvdaes/l10n or something like that), like the build-discussion action for the store. This potentially would make possible to create a branch with translations for personal repos. I can determine where to export translations on the action repo. Also, I can parse the json file with approved submitters available on the store to upload source files (readme.md and nvda.pot) for each available add-on, periodically.
Authors may tell me that they don't want for their add-ons to be uploaded, but by default this may be done, with the advantage of not having to fork repos or deals with remotes or secrets, since just my API token could be used. In this way, the project managers won't have to add invitations for authors who aren't translators to allow them to upload source files.
If an author wants to use a different project, this person can create a custom action for this using the l10nUtil.exe file. I think that this may be the bist option.