Request for Assistance with Result Reproduction

[rabi-fei]

Thank you for your excellent work!

However, we are having trouble reproducing the results. We used the meta-llama/Llama-2-13b-chat-hf model and ran the original eval_llama.sh script for reproduction. However, we observed that the speed-up ratio is only 1.05, and the acceptance ratio is 0.69. Could there be an issue with our settings?

Any guidance on this would be greatly appreciated!

[zy-ning]

My reproduction result is also kind of weird:
#Mean accepted tokens: 3.489463601532567
Token acceptance rate: 0.854723341280934
Avg Speedup: 1.0148453969005349

In theory, such high acceptance rate is expected to bring considerable speedup