onnx-coreml icon indicating copy to clipboard operation
onnx-coreml copied to clipboard

Log accurary with CoreML3

Open iiSeymour opened this issue 6 years ago • 1 comments

I was looking at adding some element-wise operations (Neg, Abs and Log) and noticed the results for Log seem off?

import onnx
import torch
import onnx_coreml
import onnxruntime
import numpy as np
from io import BytesIO

class Model(torch.nn.Module):
    def forward(self, x):
        return -x.abs().log()

buff = BytesIO()
x = torch.tensor([[1., 2., 3.], [4., 5., 6.], [7., 8., 9.]]) * 100

# run torch model
torch_model = Model()
tout = torch_model(x).numpy()

# convert torch model to coreml
torch.onnx.export(torch_model, x, buff, verbose=True)
onnx_model = onnx.load(BytesIO(buff.getvalue()))
mlmodel = onnx_coreml.convert(onnx_model, disable_coreml_rank5_mapping=True)

# run coreml model
cmlout = mlmodel.predict({'0': x.numpy()})
cout = list(cmlout.values())[0]

print("numpy \n",-np.log(x.numpy()))
print("torch \n", tout)
print("coreml\n", cout)

Using the above script to convert a torch model to coreml and comparing the results shows poor accuracy for the log operation.

graph(%0 : Float(3, 3)):
  %1 : Float(3, 3) = onnx::Abs(%0), scope: Model
  %2 : Float(3, 3) = onnx::Log(%1), scope: Model
  %3 : Float(3, 3) = onnx::Neg(%2), scope: Model
  return (%3)

1/3: Converting Node Type Abs
2/3: Converting Node Type Log
3/3: Converting Node Type Neg
Translation to CoreML spec completed. Now compiling the CoreML model.
Model Compilation done.

numpy 
 [[-4.6051702 -5.2983174 -5.7037826]
 [-5.9914646 -6.214608  -6.3969297]
 [-6.55108   -6.684612  -6.802395 ]]
torch 
 [[-4.6051702 -5.2983174 -5.7037826]
 [-5.9914646 -6.214608  -6.3969297]
 [-6.55108   -6.684612  -6.802395 ]]
coreml
 [[-4.6015625 -5.296875  -5.703125 ]
 [-5.9882812 -6.2109375 -6.3945312]
 [-6.5507812 -6.6835938 -6.8007812]]

Note: Same system info as #426.

iiSeymour avatar Jul 29 '19 17:07 iiSeymour

@aseemw should we ignore the log precision?

bhushan23 avatar Sep 05 '19 23:09 bhushan23