Experimental processes - best practices

[clausmichele]

As discussed during the meeting between Eurac and EODC yesterday, the introduction of the EOPF dependencies to extend load_stac with xcube-eopf created issues in handling the environment.
It is currently listed as a necessary dependency in pyproject.toml, which is strange since other packages necessary for load_stac are optional.

More in general, it would be good to define how experimental processes are integrated in this repo.

For instance, the rqadeforestation is an experimental processes, but it is tested in the more general test_reduce.py , which led the package rqadeforestation to be a main dependency of the project necessary to run the tests.

We should agree on some common rules, which I start drafting here:

1. New, experimental processes should be placed in the openeo_processes_dask/process_implementations/experimental folder.
2. Their dependencies must be optional.
3. Their tests should be placed in separate files (or separate experimental folder?) which can be skipped in the dependencies are not installed.
4. To avoid conflicts with experimental processes, I would propose to install them only when requested, similarly as we do for the ml here:
   https://github.com/Open-EO/openeo-processes-dask/blob/main/pyproject.toml#L64-L66

@rtmiz @ValentinaHutter @suriyahgit @Yuvraj198920 please let me know what you think about my previous points and if you think we should add some more.

Finally, since the EOPF extension of load_stac was an addition of an existing process and not a new experimental one, we also need to come up with proposal to extend an existing process with a plug-in approach.

[suriyahgit]

we can plan to move the xcube-eopf integration into a plugin-based approach, similar to what we already foresee for the defair-based native data readers in the context of the dedl project. This plugin can then be installed as an optional dependency, following the same pattern currently used for the ML components. This would keep the core environment lightweight and avoid introducing unnecessary constraints for users who do not require these extensions.

To be ensured:

1. The integration remains transparent from the user perspective (e.g., load_stac can dynamically leverage the plugin if available).
2. The plugin dependencies are fully isolated, so that installing them does not introduce conflicts with the base environment.

[clausmichele]

The plugin dependencies are fully isolated, so that installing them does not introduce conflicts with the base environment.

Is this actually possible?