Version 3.0.2

Upgrade to llama.cpp b2797

• Adds explicit support for Phi-3
• Adds flash attention
• Fixes #54

Version 3.0.1

• Updated the binding to llama.cpp b702 to add llama 3 support
• Fix #54 by using codellama for testing

Version 3.0.0

Version 3.0 updates to the newest available version of llama.cpp and all its available features. The Java binding reworks almost all of the C++ code. It heavily relies on the llama.cpp server code, which theoretically should lead to much better performance, concurrency, and long-term maintainability.

The biggest change is how model and inference parameters are handled (see examples for details). Previous versions relied on properly typed Java classes, whereas the C++ server code mostly uses JSON. The JNI code to transfer the parameters from Java to C++ was complex and error-prone. The new version comes with almost no API changes regarding how parameters are handled (apart from the available parameters per se), but should be much easier to maintain in the long term.

Version 2.3.5

Version 2.3.5

• Fixed #45: handling of special tokens in tokenizer (credit to @hvisser )

v2.3.4

Version 2.3.4

Library loading from APK on Android (Credit to @samolego )

v2.3.3

Version 2.3.3

Thanks to @samolego

• Fixed some JNI references leading to segmentation faults
• Fixed JNI wrong data type method usage

v2.3.2

Version 2.3.2

• Add android support

v2.3.1

Version 2.3.1

• Fixed macos aarch64 build
• Defaulting models.home variable

v2.3.0

Version 2.3.0

• Support for latest llama.cpp and mixtral version (thanks to @cestella )
• Extended pre-built libraries

v2.2.1

Version 2.2.1

• added more information to llm outputs (token/probabilities)
• some github workflow bug fixes