llCurious comments

Results 28 comments of


                                            llCurious

请问original weight, predictor weights是怎么生成的？

@YixinSong-e . Same question here. I notice that PowerInfer adopts a different sparsity strategy, i.e., considering the power la, while DeJa Vu seems to use threshold of 0 for sparsity....

Meta: Wider model support for PowerInfer

Hi @YixinSong-e . I notice that you provide the ReLU-LLaMA in HF. I run the model and found that the sparsity (values lower than zero) is much lower than OPT...

Meta: Wider model support for PowerInfer

Thanks for your reply. I have a question that in my understanding, ReGLU uses element-wise multiplication, which means those zero values after ReLU remain zero, theoretically yilelding same sparsity level...

Meta: Wider model support for PowerInfer

hi @hodlen . I notice that you provide ReLUFalcon-40B in the HF. Do you have the tuned ReLU-Falcon-7B weights?

[Question]: How to use int8 API (nano or ggml?) in LLAMA inference?

Thanks, i got it. BTW, do you have the README on using BigDL optimized int4/int8 quantized computation library? Maybe the use case of quantized matmul in Linear layers in models...

Lowering aten::where encounters mismatched types and crashed

Any progress? @qihqi @ManfeiBai I also encountered the same issue.

How to implement this method combinded with decoder

Any progress to fix this issue? @Guangxuan-Xiao

Low accuracy of LeNet

Hey, @jlwatson. Sry to bother you. But do you have any ideas?

Low accuracy of LeNet

Thanks to you quick reply!! I just run the model of lenet-norelu. The accuracy results seems to work fine. BTW, do you mean for other networks like secureml, minionn and...

[Question]: The number of convolutional multiplication decreases but the communication cost increases in SPU

hi, @warpoons . Interesting idea. As pointed out by @fionser , the problem is due to the **increasing amont of truncations**. According to the Winograd algorithm, the matmul is separated...