Combine Whisper.cpp & LLama.cpp in one application

[yoann0723]

Hello, ggml-org team.
First, thank you very much for the awesome whisper.cpp and llama.cpp open-source projects.
Recently, I'm developing a local AI assistant using whisper.cpp as the ASR and llama.cpp as the LLM inference engine. But whisper.cpp and llama.cpp both depend on ggml, so I'm a little confused about how to integrate them into my app.

• Can whisper.cpp and llama.cpp share the same ggml library? So that there is only one set of ggml runtime DLLs under my app.
• Or I should integrate them separately: whisper.cpp and llama.cpp use their own ggml library, so that there are two sets of ggml runtime DLLs under my app.

Looking forward to your reply.

[KitaitiMakoto]

Notice: I'm not a member of ggml-org.

I think it's possible to share the same ggml library because whisper.cpp and llama.cpp both have configuration item to use system-installed ggml library:

whisper.cpp:

whisper.cpp/CMakeLists.txt

Line 68 in 19ceec8

option(WHISPER_USE_SYSTEM_GGML "whisper: use system-installed GGML library" OFF)

llama.cpp:
https://github.com/ggml-org/llama.cpp/blob/d21a76ac38ca9d95be610f72b613c7455fa04103/CMakeLists.txt#L34

I didn't try, though.

[yoann0723]

Thank you. I also found that they used the same ggml library in the official example talk-llama, so I'd like to have a try.

[shakfu]

I just posted an overview about my about my new python library which does exactly this and more.

see: https://github.com/shakfu/cyllama