Adding docs for EOAP_CWL on CDSE. #842
Conversation
|
APIs/openEO/EOAP-CWL.qmd
Outdated
| --- | ||
|
|
||
| With openEO, it is now possible to run CWL ([Common Workflow Language](https://www.commonwl.org/)) in the `run_udf` process. This is executed on Calrissian | ||
| On the CDSE backend, this has some extra features, which are described on this page. |
There was a problem hiding this comment.
"extra" compared to what?
There was a problem hiding this comment.
I guess you mean extra in addition to what the generic openeo-geopyspark-driver docs list. But then I would swap both sentences.
Also make it more clear that the openeo-geopyspark-driver docs are generic docs: "more information" is pretty vague.
There was a problem hiding this comment.
Rephrased. Does it look good like this?
APIs/openEO/EOAP-CWL.qmd
Outdated
| - /EOAP-CWL.html | ||
| --- | ||
|
|
||
| With openEO, it is now possible to run CWL ([Common Workflow Language](https://www.commonwl.org/)) in the `run_udf` process. This is executed on Calrissian |
There was a problem hiding this comment.
"This is executed on Calrissian" looks like incomplete sentence. I'm also not sure this implementation detail must be in inttroduction
There was a problem hiding this comment.
Rephrased: "Workflows will be executed using calrissian on kubernetes."
APIs/openEO/EOAP-CWL.qmd
Outdated
|
|
||
| ## Docker images | ||
|
|
||
| Only whitelisted Docker images can be used in the cluster. Contact us through [support](https://helpcenter.dataspace.copernicus.eu/hc/en-gb/requests/new) (You might need to create an account) |
There was a problem hiding this comment.
maybe be more explicit about what kind of docker images you refer to here. I guess here it's about docker images referenced from CWL workflow documents?
APIs/openEO/EOAP-CWL.qmd
Outdated
|
|
||
| ## Memory limits | ||
|
|
||
| As of February 2026, the memory limit is set to 15GB. |
There was a problem hiding this comment.
that's the total memory limit for all concurrent containers? or is it per container?
There was a problem hiding this comment.
That is not very clear from the documentation:
When running calrissian, you must provide a limit the the number of CPU cores (--max-cores) and RAM megabytes (--max-ram) to use concurrently
or
Maximum amount of RAM to use, e.g 1048576, 512Mi or 2G. Follows k8s resource conventions
or
- Calrissian requires specifying an envelope of RAM and CPU resources to use in the cluster, so these are provided as
--max-ramand--max-cores
There was a problem hiding this comment.
Changed the limit to be on per step basis, instead of being shared over the whole process.
And clarified this wording.
| --preserve-environment=AWS_ENDPOINT_URL_S3 \ | ||
| --preserve-environment=AWS_ACCESS_KEY_ID \ | ||
| --preserve-environment=AWS_SECRET_ACCESS_KEY \ | ||
| example_workflow.cwl example_parameters.json |
There was a problem hiding this comment.
It seems a lot of mentioned options are not essential or even risky to use unless you know what you are doing. I would just have the essentials here and suggest some more advanced options seperately
There was a problem hiding this comment.
Removed 2 unneeded parameters. Annoyingly, --tmpdir-prefix and --no-read-only are needed to be able to run for me.
|
Already merged. It is starting to get more urgent |
2 participants
https://github.com/eu-cdse/openeo-cdse-infra/issues/838