`__setitem__` raises "self operand of augmented assign must be contiguous" even though it is

[obadakhalili]

Code

N = Tensor.zeros((3, 3)).contiguous()

for i in range(3):
  for j in range(3):
    N[i][j] += 1

Expectation

N to be a 3x3 matrix of 1's.

Reality

tinygrad throws RuntimeError: self operand of augmented assign must be contiguous.

[Cheapshot003]

Doesn't Tensor.zeros((3, )) just create a 1D array of size 3?

[obadakhalili]

thanks for noting this typo @Cheapshot003. however, the issue persists.

[Cheapshot003]

Can confirm the issue. It works when you do N[i, j] = N[i, j] + 1

Should be investigated by someone who knows the internals better

[obadakhalili]

that is a nice workaround but it is very slow. I think this is a separate issue of indexing being very slow.

[gordbegli]

This error likely stems from the fact that Tinygrad doesn't support loads and stores. My current understanding is that you are supposed to do everything with Tensor operations.

This could mean using a mask to accomplish your goal.

for i in range(3):
    for j in range(3):
        N += Tensor([[1 if m == i and n == j else 0 for n in range(3)] for m in range(3)])

An alternative / hack is to perform inplace assignments on a numpy array before converting it into a Tensor.

np_array = np.zeros((3, 3))

for i in range(3):
    for j in range(3):
        np_array[i, j] += 1

N = Tensor(np_array)

Here is a minimal example showing the lack of support for in-place operations, and using a mask to accomplish the same goal.

#DOESN'T WORK
t2 = Tensor([0,0]).contiguous()
t2[0] += 1
t2[1] += 1

#WORKS
t = Tensor([0,0])
t += Tensor([1,0])
t += Tensor([0,1])