Please help for GPU installation: OSError: (External) CUDA error(209), no kernel image is available for execution on the device.

[CanadianHusky]

Hello,

Can someone please help me run the code below on GPU, Nvidia GTX0180.
I get error message on GPU

OSError: (External) CUDA error(209), no kernel image is available for execution on the device.
  [Hint: 'cudaErrorNoKernelImageForDevice'. This indicates that there is no kernel image available that is suitable for the device. This can occur when a user specifiescode generation options for a particular CUDA source file that do not include the corresponding device configuration.] 
  (at ..\paddle\phi\core\platform\device\gpu\gpu_info.cc:133)

despite following all installation steps. Following code is working fine on separate CPU virtual environment.
Cuda version 12.9.1 and cudnn 9.10.2 and nvidia driver with high enough version is installed on system and GPU processing works FINE using C# PaddleSharp binding for OCR from https://github.com/sdcb/PaddleSharp
I cannot get the layout_parsing model to work in python on GPU GTX0180 and would appreciate any help. Thank you in advance.

Below Python Code works perfectly fine on separate CPU environment

from paddlex import create_model
import paddle
print(paddle.utils.run_check())
print(paddle.device.get_device())
print("Paddle compiled with CUDA:", paddle.is_compiled_with_cuda())
print("Current device:", paddle.get_device())

model_name = "PP-DocLayout_plus-L"
model = create_model(model_name=model_name)
file = "./test.png"
output = model.predict(file, batch_size=1, layout_nms=True, threshold=0.35, layout_unclip_ratio=1.0, layout_merge_bboxes_mode="small")

for res in output:
    res.print()

My environment details with clean virtual environment are:
Python Version 3.12.9
pip install paddlepaddle-gpu==3.2.0
pip install paddlex[base]

When running from GPU environment console output says that GPU is ready and API version 12.9 which matches my installed Cuda version 12.9.1 with nvidia driver 576.57:
My GPU is GTX1080 with compute capability reported as 6.1 is that the problem ? (too low ?)

Running verify PaddlePaddle program ... 
C:\Users\default\AppData\Roaming\Python\Python312\site-packages\paddle\pir\math_op_patch.py:219: UserWarning: Value do not have 'place' interface for pir graph mode, try not to use it. None will be returned.
  warnings.warn(
I1028 12:46:24.623271 14584 pir_interpreter.cc:1524] New Executor is Running ...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W1028 12:46:24.624271 14584 gpu_resources.cc:114] Please NOTE: device: 0, GPU Compute Capability: 6.1, Driver API Version: 12.9, Runtime API Version: 12.9
I1028 12:46:24.625272 14584 pir_interpreter.cc:1547] pir interpreter is running by multi-thread mode ...
PaddlePaddle works well on 1 GPU.
PaddlePaddle is installed successfully! Let's start deep learning with PaddlePaddle now.
None
gpu:0
Paddle compiled with CUDA: True
Current device: gpu:0
Model files already exist. Using cached files. To redownload, please delete the directory manually: `C:\Users\default\.paddlex\official_models\PP-DocLayout_plus-L`.
Traceback (most recent call last):
  File "f:\Projects\PT_PaddleOCR\test_layout.py", line 19, in <module>
    model = create_model(model_name=model_name)   #,device="gpu:0"
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\default\AppData\Roaming\Python\Python312\site-packages\paddlex\model.py", line 28, in create_model
    return _ModelBasedInference(
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\default\AppData\Roaming\Python\Python312\site-packages\paddlex\model.py", line 58, in __init__
    self._predictor = create_predictor(*args, **kwargs)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\default\AppData\Roaming\Python\Python312\site-packages\paddlex\inference\models\__init__.py", line 87, in create_predictor
    return BasePredictor.get(model_name)(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\default\AppData\Roaming\Python\Python312\site-packages\paddlex\inference\models\object_detection\predictor.py", line 112, in __init__
    self.pre_ops, self.infer, self.post_op = self._build()
                                             ^^^^^^^^^^^^^
  File "C:\Users\default\AppData\Roaming\Python\Python312\site-packages\paddlex\inference\models\object_detection\predictor.py", line 143, in _build
    infer = self.create_static_infer()
            ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\default\AppData\Roaming\Python\Python312\site-packages\paddlex\inference\models\base\predictor\base_predictor.py", line 301, in create_static_infer
    return PaddleInfer(
           ^^^^^^^^^^^^
  File "C:\Users\default\AppData\Roaming\Python\Python312\site-packages\paddlex\inference\models\common\static_infer.py", line 284, in __init__
    self.predictor = self._create()
                     ^^^^^^^^^^^^^^
  File "C:\Users\default\AppData\Roaming\Python\Python312\site-packages\paddlex\inference\models\common\static_infer.py", line 477, in _create
    predictor = paddle.inference.create_predictor(config)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: (External) CUDA error(209), no kernel image is available for execution on the device.
  [Hint: 'cudaErrorNoKernelImageForDevice'. This indicates that there is no kernel image available that is suitable for the device. This can occur when a user specifiescode generation options for a particular CUDA source file that do not include the corresponding device configuration.] 
  (at ..\paddle\phi\core\platform\device\gpu\gpu_info.cc:133)

Result of pip freeze on GPU environment:

aiohappyeyeballs==2.6.1
aiohttp==3.13.1
aiosignal==1.4.0
aistudio-sdk==0.3.8
annotated-types==0.7.0
anyio==4.10.0
asttokens==3.0.0
attrs==25.4.0
bce-python-sdk==0.9.46
beautifulsoup4==4.14.2
cachetools==6.2.1
certifi==2025.8.3
cffi==2.0.0
chardet==5.2.0
charset-normalizer==3.4.4
chinese_calendar==1.10.0
click==8.3.0
colorama==0.4.6
colorlog==6.10.1
comm==0.2.2
contourpy==1.3.3
cssselect==1.3.0
cssutils==2.11.1
cycler==0.12.1
dataclasses-json==0.6.7
debugpy==1.8.11
decorator==5.1.1
decord==0.6.0
distro==1.9.0
einops==0.8.1
et_xmlfile==2.0.0
executing==2.1.0
faiss-cpu==1.12.0
filelock==3.20.0
fonttools==4.60.1
frozenlist==1.8.0
fsspec==2025.9.0
ftfy==6.3.1
future==1.0.0
GPUtil==1.4.0
greenlet==3.2.4
h11==0.16.0
hf-xet==1.2.0
httpcore==1.0.9
httpx==0.28.1
httpx-sse==0.4.3
huggingface-hub==1.0.0
idna==3.10
imageio==2.37.0
imagesize==1.4.1
ipykernel==6.29.5
ipython==8.31.0
jedi==0.19.2
Jinja2==3.1.6
jiter==0.11.1
joblib==1.5.2
jsonpatch==1.33
jsonpointer==3.0.0
jupyter_client==8.6.3
jupyter_core==5.7.2
kiwisolver==1.4.9
langchain==0.3.27
langchain-community==0.3.31
langchain-core==0.3.79
langchain-openai==0.3.35
langchain-text-splitters==0.3.11
langsmith==0.4.38
lazy_loader==0.4
lxml==6.0.2
MarkupSafe==3.0.3
marshmallow==3.26.1
matplotlib==3.10.7
matplotlib-inline==0.1.7
modelscope==1.31.0
more-itertools==10.8.0
multidict==6.7.0
mypy_extensions==1.1.0
nest-asyncio==1.6.0
networkx==3.5
numpy==2.3.1
nvidia-cublas-cu12==12.9.0.13
nvidia-cuda-runtime-cu12==12.9.37
nvidia-cudnn-cu12==9.9.0.52
nvidia-cufft-cu12==11.4.0.6
nvidia-curand-cu12==10.3.10.19
nvidia-cusolver-cu12==11.7.4.40
nvidia-cusparse-cu12==12.5.9.5
nvidia-nvjitlink-cu12==12.9.86
openai==2.6.1
opencv-contrib-python==4.10.0.84
openpyxl==3.1.5
opt-einsum==3.3.0
orjson==3.11.4
packaging==25.0
paddlepaddle-gpu==3.2.0
paddlex==3.3.5
pandas==2.3.3
parso==0.8.4
pillow==11.2.1
platformdirs==4.3.6
premailer==3.10.0
prettytable==3.16.0
prompt_toolkit==3.0.48
propcache==0.4.1
protobuf==6.31.1
psutil==6.1.1
pure_eval==0.2.3
py-cpuinfo==9.0.0
pyclipper==1.3.0.post6
pycocotools==2.0.8
pycparser==2.23
pycryptodome==3.23.0
pydantic==2.12.3
pydantic-settings==2.11.0
pydantic_core==2.41.4
pyparsing==3.2.5
pypdfium2==5.0.0
python-bidi==0.6.7
python-dateutil==2.9.0.post0
python-dotenv==1.2.1
pytz==2025.2
pywin32==308
PyYAML==6.0.2
pyzmq==26.2.0
regex==2025.10.23
requests==2.32.5
requests-toolbelt==1.0.0
ruamel.yaml==0.18.16
ruamel.yaml.clib==0.2.14
safetensors==0.6.2
scikit-image==0.25.2
scikit-learn==1.7.2
scipy==1.16.2
sentencepiece==0.2.1
setuptools==80.9.0
shapely==2.1.2
shellingham==1.5.4
six==1.17.0
sniffio==1.3.1
soundfile==0.13.1
soupsieve==2.8
SQLAlchemy==2.0.44
stack-data==0.6.3
tenacity==9.1.2
threadpoolctl==3.6.0
tifffile==2025.10.16
tiktoken==0.12.0
tokenizers==0.22.1
tornado==6.4.2
tqdm==4.67.1
traitlets==5.14.3
typer-slim==0.20.0
typing-inspect==0.9.0
typing-inspection==0.4.2
typing_extensions==4.15.0
tzdata==2025.2
ujson==5.11.0
urllib3==2.5.0
wcwidth==0.2.13
yarl==1.22.0
zstandard==0.25.0

[CanadianHusky]

additional info about cuda versions from python command line:

>>> print("Compiled with CUDA:", paddle.device.is_compiled_with_cuda())
Compiled with CUDA: True
>>> print("CUDA runtime version:", paddle.version.cuda())
CUDA runtime version: 12.9
>>> print("cuDNN version:", paddle.version.cudnn())
cuDNN version: 9.9.0

My cudnn installation is in C:\Program Files\NVIDIA\CUDNN\v9.10\bin\12.9
My cudnn version on system is 9.10.2
Is this the problem ? paddle was compiled for cudnn version 9.9.0 but I have 9.10.2 installed (which is needed and compatible for the c# binding) Are the versions so sensitive ?

[CanadianHusky]

I fixed it.

I struggled for days before posting the above. Here is what needs to be done and explanation, with the hope it will help others with similar issue.
Nvidia GTX 1080 is compute capability version 6.1 (sm61 architecture)
The problem is that with my default installation above
pip install paddlepaddle-gpu==3.2.0
a runtime cuda version 12.9 with higher requirement for compute capability is installed that is too new for sm61 architecture.
Cuda 12.9 runtime works fine with c# bindings because the native runtimes for c# has multiple versions, based on sm version - see https://github.com/sdcb/PaddleSharp?tab=readme-ov-file#native-packages-%EF%B8%8F for list of Nvidia card vs sm version

To fix python version and run on GPU GTX1080 I uninstalled paddlepaddle-gpu and re-install with lower code version 11.8 using

pip install paddlepaddle-gpu==3.2.0 -i https://www.paddlepaddle.org.cn/packages/stable/cu118/

Now the result of >>> print(paddle.utils.run_check()) is

I1028 15:19:34.573431 10992 pir_interpreter.cc:1524] New Executor is Running ...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W1028 15:19:34.573431 10992 gpu_resources.cc:114] Please NOTE: device: 0, GPU Compute Capability: 6.1, **Driver API Version: 12.9, Runtime API Version: 11.8**
I1028 15:19:34.695777 10992 pir_interpreter.cc:1547] pir interpreter is running by multi-thread mode ...
PaddlePaddle works well on 1 GPU.
PaddlePaddle is installed successfully! Let's start deep learning with PaddlePaddle now.
None
gpu:0
Paddle compiled with CUDA: True
**cuDNN version: 8.9.6**
Current device: gpu:0

Based on the above information,
I installed Cuda driver 11.8 which match paddlepaddle-gpu expectation and copied DLLs into python venv path because I have multiple cuda versions now on same system.
Existing cudnn version on runtime on my system is higher (9.x) than what paddlepadde-gpu is compiled against (8.9.6) I did not have to match exact version. It worked fine.

GPU execution works fine now but this is really hard to understand at first especially for non-Chinese speakers like me.
A list of sm## version and which python wheels are compatible, could be helpful.
Is there a full list of all available wheels under /packages/stable/ that shows which minimum cuda, compute capabililty and cudnn versions are compatible with each other. Otherwise it seems a "trial and error" to find the working combination?

Great library and performance on GPU is over 15x faster than 12core CPU. Thank you for all the efforts.

Issue can be closed if there is no compatibility list for GPU architecture and runtime versions but if it is available I would love to see it.

[CanadianHusky]

One cosmetic problem exists:
During runtime this is reported:

W1028 15:58:12.967664 15896 gpu_resources.cc:243] WARNING: device: 0. The installed Paddle is compiled with CUDNN 8.9, but CUDNN version in your machine is 8.9, which may cause serious incompatible bug. Please recompile or reinstall Paddle with compatible CUDNN version.

My machine has cudnn 9.5, 9.9 and 9.10 installed. cudnn 8.9 was never installed. I adjust path and dll location manually if needed to test different versions.
paddlepaddle-gpu seems to be printing a confusing warning, not reading the existing cudnn version correctly.
Compiled version and version on my machine are reported as identical in the warning above, but they are not identical. The warning itself is justified, but the version on the machine is not detected correctly.

[changdazhou]

If there are no further exceptions, this warning can be ignored