[Bug] [Unity] [Frontend] [ONNX] Gather produced inconsistent inference results with ONNX

[Thrsu]

The ONNX model produced inconsistent inference results with ONNX when using relax to load model and obtain the results.

Actual behavior

Traceback (most recent call last):
  ...
    np.testing.assert_allclose(onnx_output[0], tvm_output, atol=1e-3, rtol=1e-3)
  File "/workplace/software/miniconda3/envs/tflite/lib/python3.8/site-packages/numpy/testing/_private/utils.py", line 1527, in assert_allclose
    assert_array_compare(compare, actual, desired, err_msg=str(err_msg),
  File "/workplace/software/miniconda3/envs/tflite/lib/python3.8/site-packages/numpy/testing/_private/utils.py", line 844, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Not equal to tolerance rtol=0.001, atol=0.001

Mismatched elements: 24 / 72 (33.3%)
Max absolute difference: 0.92875844
Max relative difference: 31.090857
 x: array([[[[0.228867, 0.563567],
         [0.655737, 0.30893 ],
         [0.737935, 0.313822]],...
 y: array([[[[0.321285, 0.733162],
         [0.199019, 0.381254],
         [0.461184, 0.895627]],...

Environment

• TVM: almost latest version of unity branch
• onnx: 1.14.0
• onnxruntime: 1.14.1

Steps to reproduce

import onnx
import numpy as np
import onnxruntime

import tvm
from tvm import relax
from tvm.relax.frontend.onnx import from_onnx

input_data = {}
input_data['data'] = np.random.rand(5, 4, 3, 2).astype(np.float32)
input_data['indices'] = np.random.randint(-2, 3, [3], dtype=np.int64)

onnx_model_path = "Gather.onnx"
model = onnx.load(onnx_model_path)

sess = onnxruntime.InferenceSession(model.SerializeToString())
onnx_output = sess.run(None, input_data)

tvm_model = from_onnx(model, opset=18, keep_params_in_input=True)
tvm_model = relax.transform.DecomposeOpsForInference()(tvm_model)
tvm_model = relax.transform.LegalizeOps()(tvm_model)

tvm_model, params = relax.frontend.detach_params(tvm_model)
with tvm.transform.PassContext(opt_level=3):
    ex = relax.build(tvm_model, target="llvm")
    vm = relax.VirtualMachine(ex, tvm.cpu())

inputs = {}
inputs['data'] =  tvm.nd.array(input_data['data'].astype(np.float32))
inputs['indices'] = tvm.nd.array(input_data['indices'].astype(np.int64))
input_tvm = [
    inputs[key.name_hint] for key in tvm_model["main"].params if key.name_hint in inputs
]

if params:
    input_tvm += params["main"]

vm.set_input("main", *input_tvm)
vm.invoke_stateful("main")
tvm_output = vm.get_outputs("main").numpy()

np.testing.assert_allclose(onnx_output[0], tvm_output, atol=1e-3, rtol=1e-3)

Triage

• needs-triage

cc @KJlaccHoeUM9l @quic-sanirudh