FNsi

Replace vram only so it's very useful to generate large size img or batchs ):

Add avx-512 support?

6

No clue but I think it may work faster

[fixed]The last code build with memory fix running result is not good in my pc.

10

Be obviously slower with Q_1 30b model. And the memory usage become garbage... (Linux 5.19 x64 Ubuntu base)

Infinity transcript mode may stuck in ram?

1

After ctx > 2048 or whatever set in -c, While close the terminal, the transcript may have a chance continuously running in system. Linux amd64 5.19 ubuntu base.

Little confuse in quantize.cpp

2

Line 13-20 in quanzize.cpp, The printf function is not match the sequnce in current master e0305ea. While ggml.h and llama.h are using different numbers to call quantisations.

A simple script merge lora to HF

4

Thanks for #1516 I hope your guys enjoy it.

[Feature Request] ONNX support

61

In compact structure (model size 256k~4m) that would be a runtime effect base on DirectMl Am I so greedy?😂

[Feature Request]Downgrade PyTorch to 2.1.0 to Add PyTorch-directml backend

3

😅 meanwhile python version seems need to be 3.10

Only few people will face it.(who cannot download it automatically but copy their own ffmpeg.exe to the location)😂😭 1' not checking whether ffprobe.exe exist or not when ffmpwg.exe exist. 2'...

Can't download integrated python in windows.

7

99% sure it's base on my shit "internet" May you offer a direct link for the whole python stuff or allow to use an local python?