llama.cpp icon indicating copy to clipboard operation
llama.cpp copied to clipboard

Published 20 hours ago • verified publisher icon ggml-org

examples : evaluate tokens in batches after swapping context

Open grencez opened this issue 1 year ago • 1 comments

This new loop around llama_eval is a bit redundant with the batching done in the main loop, but without a refactor it's all still necessary to keep print statements happening at the right times.

grencez avatar Apr 16 '23 09:04 grencez

Tests passed yesterday. I just synced recent changes and added a comment.

grencez avatar Apr 17 '23 20:04 grencez