torch

torch copied to clipboard

Tensorized Matrices vs Factorized Tensors

1

Hi, Thanks for creating and maintaining this library. I had a couple of basic questions, would be great if you could answer: 1. What is the difference between the files...

ashim95

malbergo

Support for ONNX export

1

Thank you for the library. Have you considered adding support for exporting factorized layers (`FactorizedConv`) to ONNX?

mfl22

`tltorch.FactorizedConv.from_conv` tries to allocate ~83GB memory for an input of shape (1024, 512, 3, 3, 3)

2

Minimal Code to reproduce the error: ```python import torch import tltorch test_conv3d = torch.nn.Conv3d(1024, 512, (3,3,3), padding=(3,1,1)) print(tltorch.FactorizedConv.from_conv(test_conv3d, rank='same', factorization='cp')) ``` Error: ``` RuntimeError: [enforce fail at alloc_cpu.cpp:83] err ==...

hello-fri-end

Factorized Tensor slower than Neural Network Layer !!!

3

``` import tltorch import torch from torch.profiler import profile, record_function, ProfilerActivity data = torch.randn((4, 16), dtype=torch.float32) linear = torch.nn.Linear(16, 10) fact_linear = tltorch.FactorizedLinear.from_linear(linear, auto_tensorize=False, in_tensorized_features=(4, 4), out_tensorized_features=(2, 5), rank=0.1, factorization="tucker")...

KaidDuong

Tensorized Matrices Error: np.unravel_index can't convert cuda device type tensor to numpy

1

KaidDuong

`torch.jit.script` does not work with Tensorized Models

1

Minimal Code: ```python import torch from torch.nn import Module from tltorch import FactorizedConv class Test(Module): def __init__(self): super(Test, self).__init__() self.layer = FactorizedConv(3, 4, 3, factorization='tucker', order=3) def main(): # Instantiate...

hello-fri-end

padding="same" and groups=in_channels in 3D convolution not supported

1

I have tried to make a 3D convolutional layer with the following parameters: `padding='same'` and `groups='in_channels' ` and l get an `AssertionError.` Either with `tltorch.FactorizedConv()` or `tltorch.FactorizedConv.from_conv()` It seems that...

GDLmazari