Jesse Cai comments

Results 46 comments of


                                            Jesse Cai

[sparse] add search for optimal alg_id to torch.compile

@pytorchbot rebase

[sparse] add search for optimal alg_id to torch.compile

@pytorchbot merge

[sparse] add search for optimal alg_id to torch.compile

@pytorchbot merge -f "unrelated failure"

[sparse] add search for optimal alg_id to torch.compile

@pytorchbot revert

[sparse] add search for optimal alg_id to torch.compile

@pytorchbot revert -m "this PR breaks AO tests" -c nosignal

[sparse] add search for optimal alg_id to torch.compile

@pytorchbot rebase

[sparse] add search for optimal alg_id to torch.compile

@pytorchbot rebase

[sparse] add search for optimal alg_id to torch.compile

@pytorchbot merge -f "unrelated failure on cuDNN path"

`int4_weight_only` Slows Down `torch.nn.Linear` for Llama2 7B Shapes

cc @mostafaelhoushi What hardware, pytorch and ao version are you using? On my H100 on the nightlies, I see: (for 4096, 11008) which is a speedup ``` Baseline: 0.04709856033325195 ms...

`int4_weight_only` Slows Down `torch.nn.Linear` for Llama2 7B Shapes

I would definitely recommend using the latest nightlies to test