Conversation
|
Hi @idales thank you very much for this PR! I'll look at it over the weekend and merge it. Until then I'll try to approve running the pipeline. |
|
Hi @idales building the C++ code locally, I get the same error as the CI runners |
|
I was able to pass it by passing modelParameters.setNthreads(8) so instead of default value. But i have another error that I am getting and looking into it. Error: libc++abi: terminating due to uncaught exception of type nlohmann::json_abi_v3_11_3::detail::type_error: [json.exception.type_error.302] type must be number, but is array |
Awesome, nice hint, thanks!
The Java library has the two classes |
|
Looks like it's tensor_split ( abetlen/llama-cpp-python#1016 ) but i don't see it set anywhere and i even removed ( commented ) [] brackets in the code. |
| { | ||
| gpt_params params; | ||
| common_params params; | ||
|
|
||
| auto *ctx_server = new server_context(); | ||
|
|
||
| std::string c_params = parse_jstring(env, jparams); | ||
| json json_params = json::parse(c_params); | ||
| server_params_parse(json_params, params); | ||
|
|
||
| if (json_value(json_params, "disable_log", false)) | ||
| const jsize argc = env->GetArrayLength(jparams); | ||
| char **argv = parse_string_array(env, jparams, argc); | ||
| if (argv == nullptr) | ||
| { | ||
| log_disable(); | ||
| } | ||
| else | ||
| { | ||
| log_enable(); | ||
| return; | ||
| } | ||
|
|
||
| if (!params.system_prompt.empty()) | ||
| const auto parsed_params = common_params_parse(argc, argv, params, LLAMA_EXAMPLE_SERVER); | ||
| free_string_array(argv, argc); | ||
| if (!parsed_params) | ||
| { | ||
| ctx_server->system_prompt_set(params.system_prompt); | ||
| return; | ||
| } |
There was a problem hiding this comment.
@vaiju1981 I changed how the model parameters are passed from Java to C++. The code now assembles a CLI string (e.g. something like --model /path/to/file -c 2048). This way we can better re-use the existing llama.cpp code to parse the parameters and don't have to rely on custom logic. So far it seems to work great, but I'm now stuck after requesting a completion and getting no output. I'll investigate further over the next few days.
Hi @kherud |
|
I am able to pass all the test except for embedding. I will keep you posted if I can fix it. Right now the only issue is with embedding call: libc++abi: terminating due to uncaught exception of type nlohmann::json_abi_v3_11_3::detail::type_error: [json.exception.type_error.302] type must be number, but is array |
|
How important are testLogJSON and testLogText tests, apart from these and embedding i am able to run all the other tests. |
|
I'd say if logging in general works don't worry about the tests. Logging is tricky to test, so the tests were never really reliable anyway. Feel free to shoot a pull request 👍 |
|
apart from logging, rest is working fine, I also updated to the latest release tag of llama.cpp. I created new PR: #93 It kept on giving me permission error when updating this branch. |
|
Thank you again for this PR. We finally finished #93 and just released a new major version to upgrade to the newest available llama.cpp version. Thus I'll close this PR. |
3 participants
support for the b4513 version of the llama.cpp