initial Intel GPU support for whisper.cpp with SYCL and BLAS by izelnakri · Pull Request #2 · kilyanni/intel-nix

izelnakri · 2025-08-12T18:18:39Z

My initial milestone of 3-4 months ago, now even better with BLAS 😅 🎉 🎉

Working flawlessly real-time on my Lunar Lake GPU!!

src/ggml/whisper-cpp.nix

kilyanni · 2025-08-13T16:19:21Z

src/ggml/whisper-cpp.nix

+  postInstall = ''
+    for binary in $out/bin/*; do
+      if [[ -f "$binary" && -x "$binary" ]]; then
+        wrapProgram "$binary" \
+          --prefix LD_LIBRARY_PATH : "/run/opengl-driver/lib:/run/opengl-driver-32/lib:${openvino}/runtime/lib/intel64"
+      fi
+    done
+  '';


Does autoAddDriverRunpath work, or does it cause issues with openvino?

Actually I'm not sure openvino part works yet. Is autoAddDriverRunpath has some advantages over wrapProgram or less costs or is it the convention? I've also seen it in the driver packaging so far.

It's the convention for GPU stuff (especially with CUDA). Wrappers also add (minor) unneeded overhead compared to patching (which autoAddDriverRunpath does). Also, autoAddDriverRunpath is a oneliner (just add to nativeBuildInputs) as opposed to this wrapping

Did you try autoAddDriverRunpath? Did it work?

I can't really test because AMDGPU works differently

I'm trying to improve nixpkgs#openvino first, so I wont even need LD_LIBRARY_PATH for it hopefully, then will look into this.

Hi again @blenderfreaky I've added autoAddDriverRunpath but I now get these runtime errors few seconds after booting up the whisper.cpp:

[Start speaking] [BLANK_AUDIO]level_zero backend failed with error: 20 (UR_RESULT_ERROR_DEVICE_LOST) Exception caught at file:/build/source/ggml/src/ggml-sycl/ggml-sycl.cpp, line:3930, func:operator() SYCL error: CHECK_TRY_ERROR((stream)->wait()): Exception caught in this line of code. in function ggml_backend_sycl_synchronize at /build/source/ggml/src/ggml-sycl/ggml-sycl.cpp:3930 /build/source/ggml/src/ggml-sycl/../ggml-sycl/common.hpp:126: SYCL error [New LWP 23765] [New LWP 23721] [New LWP 23720] [New LWP 23715] [New LWP 23712] [New LWP 23710] warning: Loadable section ".dynstr" outside of ELF segments in /nix/store/fr456rbg61nnnfq6z9ffbg64llf111x4-whisper-cpp/bin/whisper-stream warning: File "/nix/store/fkw48vh7ivlvlmhp4j30hy2gvg00jgin-gcc-14.3.0-lib/lib/libstdc++.so.6.0.33-gdb.py" auto-loading has been declined by your `auto-load safe-path' set to "$debugdir:$datadir/auto-load:/nix/store/dj06r96j515npcqi9d8af1d1c60bx2vn-gcc-14.3.0-lib". To enable execution of this file add add-auto-load-safe-path /nix/store/fkw48vh7ivlvlmhp4j30hy2gvg00jgin-gcc-14.3.0-lib/lib/libstdc++.so.6.0.33-gdb.py line to your configuration file "/home/izelnakri/.config/gdb/gdbinit". To completely disable this security protection add set auto-load safe-path / line to your configuration file "/home/izelnakri/.config/gdb/gdbinit". For more information about this security protection see the "Auto-loading safe path" section in the GDB manual. E.g., run from the shell: info "(gdb)Auto-loading safe path" warning: File "/nix/store/7yk44bwycl6pfd15wrgj5pm6f9vwd5c0-intel-llvm-nightly-2025-08-05/lib/libsycl.so.8.0.0-0-gdb.py" auto-loading has been declined by your `auto-load safe-path' set to "$debugdir:$datadir/auto-load:/nix/store/dj06r96j515npcqi9d8af1d1c60bx2vn-gcc-14.3.0-lib". [Thread debugging using libthread_db enabled] Using host libthread_db library "/nix/store/g8zyryr9cr6540xsyg4avqkwgxpnwj2a-glibc-2.40-66/lib/libthread_db.so.1". 0x00007f212310a87b in wait4 () from /nix/store/lmn7lwydprqibdkghw7wgcn21yhllz13-glibc-2.40-66/lib/libc.so.6 #0 0x00007f212310a87b in wait4 () from /nix/store/lmn7lwydprqibdkghw7wgcn21yhllz13-glibc-2.40-66/lib/libc.so.6 #1 0x00007f2123d732e6 in ggml_print_backtrace () from /nix/store/fr456rbg61nnnfq6z9ffbg64llf111x4-whisper-cpp/lib/libggml-base.so #2 0x00007f2123d72912 in ggml_abort () from /nix/store/fr456rbg61nnnfq6z9ffbg64llf111x4-whisper-cpp/lib/libggml-base.so #3 0x00007f21238410cb in ggml_sycl_error(char const*, char const*, char const*, int, char const*) () from /nix/store/fr456rbg61nnnfq6z9ffbg64llf111x4-whisper-cpp/lib/libggml-sycl.so #4 0x00007f212386f3d1 in ggml_backend_sycl_synchronize(ggml_backend*) () from /nix/store/fr456rbg61nnnfq6z9ffbg64llf111x4-whisper-cpp/lib/libggml-sycl.so #5 0x00007f2123d8cb1b in ggml_backend_sched_graph_compute () from /nix/store/fr456rbg61nnnfq6z9ffbg64llf111x4-whisper-cpp/lib/libggml-base.so #6 0x00007f2123f7bd65 in ggml_graph_compute_helper(ggml_backend_sched*, ggml_cgraph*, int, bool) () from /nix/store/fr456rbg61nnnfq6z9ffbg64llf111x4-whisper-cpp/lib/libwhisper.so.1 #7 0x00007f2123f75072 in whisper_encode_internal(whisper_context&, whisper_state&, int, int, bool (*)(void*), void*) () from /nix/store/fr456rbg61nnnfq6z9ffbg64llf111x4-whisper-cpp/lib/libwhisper.so.1 #8 0x00007f2123f7e0c5 in whisper_full_with_state () from /nix/store/fr456rbg61nnnfq6z9ffbg64llf111x4-whisper-cpp/lib/libwhisper.so.1 #9 0x00007f2123f89aeb in whisper_full () from /nix/store/fr456rbg61nnnfq6z9ffbg64llf111x4-whisper-cpp/lib/libwhisper.so.1 #10 0x00000000004134ea in main () [Inferior 1 (process 23699) detached] ^C^C^C^C^C^C^C^C^C^C^Czsh: abort (core dumped) ./result/bin/whisper-stream -m ~/Github/whisper.cpp/models/ggml-base.en.bin -

Granted I didnt rebase with master yet, so could be that src.llvm packages aren't with autoAddDriverRunpath on my branch and this might be causing it. Regardless since this is the only stable version(unless I get a confirmation from you that we can try with rebasing master now), lets merge this PR and try to remove this block later.

kilyanni · 2025-08-19T02:55:09Z

src/openvino.nix

+    pugixml
+    snappy
+    tbb_2022
+    tree


[nit] This seems like a debugging leftover?

Suggested change

tree

kilyanni · 2025-08-19T02:55:22Z

src/openvino.nix

+  tbb_2022,
+  cudaPackages,
+
+  tree


Suggested change

tree

kilyanni · 2025-08-19T02:58:26Z

src/openvino.nix

+  # prevent scons from leaking in the default python version
+  scons' = scons.override { inherit python3Packages; };
+
+  tbbbind_version = "2_5";


is this not provided by our oneTBB derivation? The docs claim this should be included

(didn't mean to reference scons', only tbbbind)

I prefer to take small steps, didn't manage to try to remove/upgrade this. Code is mainly from nixpkgs with improvements. Better to do this one in an another PR perhaps for stability

kilyanni · 2025-08-19T02:59:41Z

src/openvino.nix

+    "-DOpenCV_DIR=${lib.getLib opencv}/lib/cmake/opencv4/"
+    "-DProtobuf_LIBRARIES=${protobuf}/lib/libprotobuf${stdenv.hostPlatform.extensions.sharedLibrary}"
+    "-DPython_EXECUTABLE=${python.interpreter}"


Does it fail to auto-discover if you put python in nativeBuildInputs?
For python for example they seem to search normally, see here

kilyanni · 2025-08-19T03:06:57Z

src/openvino.nix

+  postFixup = ''
+    # Link to OpenCL
+    find $out -type f \( -name '*.so' -or -name '*.so.*' \) | while read lib; do
+      addDriverRunpath "$lib"
+    done
+  '';


Suggested change

postFixup = ''

# Link to OpenCL

find $out -type f $ -name '*.so' -or -name '*.so.*' $ | while read lib; do

addDriverRunpath "$lib"

done

'';

nativeBuildInputs = [

...

autoAddDriverRunpath

];

The hook should do basically the same thing

initial Intel GPU support for whisper.cpp with SYCL and BLAS

5b969d9

kilyanni reviewed Aug 13, 2025

View reviewed changes

src/ggml/whisper-cpp.nix Outdated Show resolved Hide resolved

kilyanni reviewed Aug 13, 2025

View reviewed changes

src/ggml/whisper-cpp.nix Outdated Show resolved Hide resolved

kilyanni reviewed Aug 13, 2025

View reviewed changes

blenderfreaky requested changes to whisper.cpp except openvino

ae233dc

izelnakri force-pushed the sycl-whisper-cpp branch from be06b89 to 6281df4 Compare August 18, 2025 12:52

Add improved openvino nix package

f17e57d

izelnakri force-pushed the sycl-whisper-cpp branch from 6281df4 to f17e57d Compare August 18, 2025 16:26

Improved whisper-cpp after openvino improvements

94daaf8

kilyanni reviewed Aug 19, 2025

View reviewed changes

src/openvino.nix

tbb_2022,

cudaPackages,

tree

Copy link

Owner

kilyanni Aug 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change

tree

kilyanni reviewed Aug 19, 2025

View reviewed changes

Conversation

izelnakri commented Aug 12, 2025

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kilyanni Aug 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

kilyanni Aug 19, 2025 •

edited

Loading