Segmentation Fault on applications that use PyQt5 and the xcube plugin system

[Hneuschmidt]

Describe the bug

When developing a PyQt5 application, accessing functionality of xcube that relies on xcube plugins results in a segmentation fault.
The line in xcube where the segfault occurs appears to be this call to importlib.load_module

xcube/xcube/util/extension.py

Line 356 in d96967e

module = importlib.import_module(module_name)

.

The faulthandler standard library module gives more useful insight. The full faulthandler output can be found in faulthandler.txt

Fatal Python error: Segmentation fault

Current thread 0x00007f01fb618740 (most recent call first):
  File "/home/hannes/temp/20250314/importlib_bug_mre/env/lib/python3.12/site-packages/llvmlite/binding/ffi.py", line 197 in __call__
  File "/home/hannes/temp/20250314/importlib_bug_mre/env/lib/python3.12/site-packages/llvmlite/binding/executionengine.py", line 99 in finalize_object
  File "/home/hannes/temp/20250314/importlib_bug_mre/env/lib/python3.12/site-packages/numba/core/codegen.py", line 1068 in wrapper
...
  File "/home/hannes/temp/20250314/importlib_bug_mre/env/lib/python3.12/importlib/__init__.py", line 90 in import_module
  File "/home/hannes/temp/20250314/importlib_bug_mre/env/lib/python3.12/site-packages/xcube/util/extension.py", line 356 in _import_component
...

This is likely caused by an incompatibility issue between PyQt5 and numba (which is used by xcube) and not by xcube itself (see Additional Context below). It does, however, affect users that want to use xcube and PyQt5 in the same project.

To Reproduce

Steps to reproduce the behavior:

1. Create a conda environment with xcube, xcube-stac and pyqt

micromamba create -p ./brokenenv xcube=1.8 xcube-stac=0.2 pyqt=5.15

2. Activate the environment

micromamba activate ./brokenenv

3. Write a script that creates a PyQt5 application and a window and accesses some functionality of xcube-stac:

from PyQt5.QtWidgets import  QApplication, QWidget
from xcube.core.store import get_data_store_params_schema

import sys
import faulthandler

def main(args=None):
    faulthandler.enable(all_threads=True)
    app = QApplication(args or [])
    window = QWidget()
    window.show()
    s = get_data_store_params_schema("stac-xcube")
    print(s.to_dict())

    #app.exec() # not even needed

    return 0

if __name__ == "__main__":
    sys.exit(main())

4. Execute the script

python main.py

Expected behavior

I see a window appear (from QT) and a dictionary representation of the data store schema printed to the console

Observed behavior

A window appears but the script crashes with a segmentation fault and does not print anything to the console.

Additional context

I have traced the problem to an incompatibility between pyqt and numba. Numba and newer versions of pyqt depend on libllvm. Unfortunately, versions of libllvm appear as separate packages, for example libllvm19 and libllvm15 which are the two versions present in the environment created above.

Therefore, conda does not solve the environment to only include one version of libllvm. I believe that the incompatibility is caused by the presence of two (apparently incompatible) versions of libllvm in the environment.

The bug can be reproduced without the involvement of xcube with the following two scripts:

# numbamodule.py
import numba

@numba.jit()
def add(a, b):
    return a + b

# main2.py
import importlib

from PyQt5.QtWidgets import  QApplication, QWidget

import faulthandler
import sys

def main():
    faulthandler.enable(all_threads=True)
    app = QApplication([])
    window = QWidget()
    window.show()
    p = importlib.import_module("numbamodule")
    res = p.add(1, 2)
    print(res)


    return 0

if __name__ == "__main__":
    sys.exit(main())

Note that it is necessary to dynamically import the module.
If the we change from using importlib.import_module to simply using an import statement, the same issue appears. Importing the module outside the main function at the top of the file or simply using numba in the main2.py script does not cause the segmentation fault.

Partial Workaround

The issue with numba and pyqt can be avoided by installing an older version of pyqt that does not depend on any version of libllvm, such as pyqt=5.9.

# works
micromamba -p ./env_fixed create python numba pyqt=5.9

Unfortunately, pyqt version 5.9 requires Python <3.8 which is beyond its end of life and the minimum required Python version of xcube-stac is Python 3.10, so that this fix does not work for xcube and xcube-stac.

[Hneuschmidt]

It appears this issue affects other projects that use llvm such as Julia and Pytorch and is being tracked by conda-forge conda-forge/llvmlite-feedstock#99 and there is a pull request conda-forge/llvmlite-feedstock#100 ("merge in the next 1-2 weeks", reference: 2025-03-14) to change to static linking of libllvm to avoid the issues caused by conflicting libllvms present in conda environments.