Investigate why Unispac/Llama2-7B-Chat-Augmented is so slow #24
Description
Unispac/Llama2-7B-Chat-Augmented is very slow in generation with generate_ragged_batched - would be nice to understand why.
{{ message }}
Unispac/Llama2-7B-Chat-Augmented is very slow in generation with generate_ragged_batched - would be nice to understand why.