iree icon indicating copy to clipboard operation
iree copied to clipboard

Provide more extensive support for collapse into padded tensor in codegen or generalize checks for when we can safely add pad encodings

Open jtuyls opened this issue 6 months ago • 0 comments

Sometimes when we add pad encodings, the blocked dimensions can make it hard to fold the tensor.collapse_shape and the iree_tensor_ext.dispatch.tensor.store operations, for example:

%collapsed = tensor.collapse_shape %36 [[0], [1, 2, 3]] : tensor<4x8x4x128xf8E4M3FNUZ> into tensor<4x4096xf8E4M3FNUZ>
iree_tensor_ext.dispatch.tensor.store %collapsed, %21, offsets = [0, 0], sizes = [4, 4096], strides = [1, 1] : tensor<4x4096xf8E4M3FNUZ> -> !iree_tensor_ext.dispatch.tensor<writeonly:tensor<4x4224xf8E4M3FNUZ>>

Currently, codegen only handles the case when the padded dimension is a multiple of the blocked dimensions. This needs some investigation on how to best generalize this towards more cases and/or add checks to avoid getting into this case.

jtuyls avatar Jun 20 '25 07:06 jtuyls