Git: master: AI object mask tool : NVIDIA CUDA not available, will fall back to CPU

[KarlMagnusLarsson]

Is there an existing issue for this?

• I checked and did not find my issue in the already reported ones

Describe the bug

Hello @andriiryzhkov, Thank you for the AI object mask tool in #20378 (comment). I have tried it since it is now in darktable-org:master. It works for me using CPU.

However, if I pick NVIDA CUDA I get NVIDIA CUDA not available, will fall back to CPU.

Steps to reproduce

1. Build darktable from source with new CMAKE parameters -DUSE_AI=ON -DBUILD_CURVE_TOOLS=ON -DBUILD_NOISE_TOOLS=ON

git clone https://github.com/darktable-org/darktable.git
cd darktable
git submodule update --init
mkdir build && cd build
cmake -DCMAKE_INSTALL_PREFIX=/home/magnus/test/darktable-test/install -DCMAKE_BUILD_TYPE=Release -DUSE_AI=ON -DBUILD_CURVE_TOOLS=ON -DBUILD_NOISE_TOOLS=ON ..
make -j32
make install

2. run darktable

./install/bin/darktable --configdir /home/magnus/test/darktable-test/config --cachedir /home/magnus/test/darktable-test/cache

3. preferences -> AI -> download all
4. preferences -> AI -> enable AI features
5. preferences -> AI -> execution provider -> NVIDIA CUDA

I get the error described in describe the bug above 'NVIDIA CUDA not available, will fall back to CPU'

I have CUDA 12.4 active and other darktable functions do use the GPU, but I can not get AI object mask tool to operate on the GPU.

What am I missing to get NVIDA CUDA working?

Expected behavior

Find end execute CUDA. Document prerequisites for AI object mask tool using NVIDIA CUDA (CMAKE, librariers, configuration)

Logfile | Screenshot | Screencast

No response

Commit

85e1d43

Where did you obtain darktable from?

self compiled

darktable version

release-5.5.0-605-g85e1d43e56

What OS are you using?

Linux

What is the version of your OS?

Debian 13.2 Trixie

Describe your system

• Debian 13.3
• CPU Intel 14900K
• GPU Nvidia Quadro RTX 4000, 8GB
• 64 6B RAM
• X11
• NVIDIA Driver Version: 550.163.01
• CUDA 12.4

$ apt list *cuda* --installed
libcuda1/stable,now 550.163.01-2 amd64 [installed,automatic]
libcudart12/stable,now 12.4.127~12.4.1-2 amd64 [installed,automatic]
nvidia-cuda-dev/stable,now 12.4.127~12.4.1-2 amd64 [installed]
nvidia-cuda-gdb/stable,now 12.4.127~12.4.1-2 amd64 [installed,automatic]
nvidia-cuda-toolkit-doc/stable,now 12.4.1-2 all [installed,automatic]
nvidia-cuda-toolkit-gcc/stable,now 12.4.1-2 amd64 [installed]
nvidia-cuda-toolkit/stable,now 12.4.131~12.4.1-2 amd64 [installed]

$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2024 NVIDIA Corporation
Built on Thu_Mar_28_02:18:24_PDT_2024
Cuda compilation tools, release 12.4, V12.4.131
Build cuda_12.4.r12.4/compiler.34097967_0

EDIT: Compute capability 7.5 (if that matters)

Found 1 device(s).
Device: 0
  Name: Quadro RTX 4000
  Compute Capability: 7.5
  Multiprocessors: 36
  CUDA Cores: 2304

Are you using OpenCL GPU in darktable?

Yes

If yes, what is the GPU card and driver?

GPU Nvidia Quadro RTX 4000, 8GB , NVIDIA Driver Version: 550.163.01

Please provide additional context if applicable. You can attach files too, but might need to rename to .txt or .zip

No response

[TurboGit]

Same for me on GNU/Debian/sid, so as for the macOS packaged libonnxruntime the GNU/Linux package is not compiled with GPU acceleration support.

[KarlMagnusLarsson]

I had libonnxruntime-dev/stable 1.21.0+dfsg-1 amd64 installed on Debian Trixie 13.3. I removed the package and recompiled darktable. Then I got an ONNX runtime download during the build:

-- Downloading ONNX Runtime 1.23.2 (onnxruntime-linux-x64-1.23.2.tgz)...
-- [download 0% complete]
-- [download 1% complete]
-- [download 2% complete]
-- [download 3% complete]
-- [download 4% complete]
-- [download 5% complete]
-- [download 6% complete]
-- [download 7% complete]
-- [download 8% complete]
-- [download 9% complete]
-- [download 10% complete]
-- [download 11% complete]
-- [download 12% complete]
-- [download 13% complete]
-- [download 14% complete]
-- [download 15% complete]
-- [download 16% complete]
...

But with the same issue, fallback to CPU.

[TurboGit]

Right, looks like there is a specific archive to download for GPU. I'll propose a PR, but I'll let @andriiryzhkov review as I'm not sure this is the correct way.

[TurboGit]

@KarlMagnusLarsson : See #20522

[KarlMagnusLarsson]

Thank you @TurboGit. I works for me after a very shallow test: No warnings and GPU activity eyeballing nvidia-smi output.

I can use both Sam2.1 and SegNext models (also I have a small GPU but with 8 GB RAM) and I do not get NVIDIA CUDA not available, will fall back to CPU warning anymore.

[andriiryzhkov]

Solution from #20522 should work fine. Probably having system packages with GPU support would be better, but unfortunately not yet available.

Just notice that AI mask will anyway default to CPU, because moving computational graphs to GPU costs more time than inference on CPU itself.