llama.cpp Streaming conversion with no torch

Streaming conversion with no torch

Open diimdeep opened this issue 1 year ago • 3 comments

Drop torch, do not load whole file into memory, process files in parallel and use separate threads for r/w

Mar 15 '23 20:03 diimdeep

The python dependencies in .devops/full.Dockerfile should also be updated, will conflict with my PR #293.

Mar 19 '23 14:03 sw

This looks like a very useful addition. Lets give it a priority and merge after resolving the conflicts

Mar 19 '23 20:03 ggerganov

@ggerganov Any update on this? Because I really do not want to install pytorch on my system (because of memory).

Mar 26 '23 09:03 tim-gromeyer

This is probably too outdated so closing for now

Mar 30 '23 19:03 ggerganov