Convert model to CoreML

[tama4ma]

System info ・torch 1.12.1 ・coremltools 6.1 ・python 3.8.15

I tried to convert model of votenet to the Core ML format. I am referring to the following. https://coremltools.readme.io/docs/pytorch-conversion

After defining the model I get an error like below when converting to 'torchscript'

RuntimeError                              Traceback (most recent call last)
[<ipython-input-37-277f5d3f2bc7>](https://localhost:8080/#) in <module>
----> 1 trace = torch.jit.trace(model, dummy_input)

1 frames
[/usr/local/lib/python3.8/dist-packages/torch/jit/_trace.py](https://localhost:8080/#) in trace(func, example_inputs, optimize, check_trace, check_inputs, check_tolerance, strict, _force_outplace, _module_class, _compilation_unit)
    748 
    749     if isinstance(func, torch.nn.Module):
--> 750         return trace_module(
    751             func,
    752             {"forward": example_inputs},

[/usr/local/lib/python3.8/dist-packages/torch/jit/_trace.py](https://localhost:8080/#) in trace_module(mod, inputs, optimize, check_trace, check_inputs, check_tolerance, strict, _force_outplace, _module_class, _compilation_unit)
    965             example_inputs = make_tuple(example_inputs)
    966 
--> 967             module._c._create_method_from_trace(
    968                 method_name,
    969                 func,

RuntimeError: Encountering a dict at the output of the tracer might cause the trace to be incorrect, this is only valid if the container structure does not change based on the module's inputs. Consider using a constant container instead (e.g. for `list`, use a `tuple` instead. for `dict`, use a `NamedTuple` instead). If you absolutely need this and know the side effects, pass strict=False to trace() to allow this behavior.

This is my code

model = VoteNet(10,12,10,np.random.random((10,3))).to('cuda')
model.eval()
model.load_state_dict(torch.load('checkpoint.tar'), strict=False) 
dummy_input = {'point_clouds': torch.rand((20000,3)).unsqueeze(0).cuda()}
trace = torch.jit.trace(model, dummy_input)

I searched problem,then I found this https://github.com/huggingface/transformers/issues/9095

I changed code like this based on comments in the issue

model = VoteNet(10,12,10,np.random.random((10,3))).to('cuda')
model.eval()
model.load_state_dict(torch.load('checkpoint.tar'), return_dict=False) 
dummy_input = {'point_clouds': torch.rand((20000,3)).unsqueeze(0).cuda()}
trace = torch.jit.trace(model, dummy_input)

But this time I got the following error

TypeError                                 Traceback (most recent call last)
[<ipython-input-31-0811dab20119>](https://localhost:8080/#) in <module>
----> 1 model.load_state_dict(torch.load('checkpoint.tar'), return_dict= False)

TypeError: load_state_dict() got an unexpected keyword argument 'return_dict'

I'm a beginner, so maybe I'm stumbling on a rudimentary part Could you help me?

[ghost]

--export-type <type>, available types are ply, obj, glb and gltf.

print("# Step 9: Texture Mesh")
pTexture = subprocess.Popen([os.path.join(OPENMVS_BIN, "TextureMesh"),
os.path.join(reconstruction_dir, "scene_dense_mesh_refine.mvs"), "--export-type", "<type>"])
pTexture.wait()

[elMerzoukilzk]

Hi @4CJ7T, thank you for your response. I have received the following files: scene_dense_mesh_refine_texture.png, scene_dense_mesh_refine_texture.ply, scene_dense_mesh_refine_texture.mvs, and scene_dense_mesh_refine_texture.gltf. Is it possible to combine the mesh and texture into a single file that contains the complete 3D model, without the need for separate png and mesh files? Thank you.

[ghost]

Not in OpenMVS currently, the glTF type was implemented before glTF2, which added the option of storing textures in the file.

[elMerzoukilzk]

Hi @4CJ7T , thank you for responding. Could you please explain how to embed the texture directly within the GLB file instead of using references? If you have any ideas or Python code (not software) that can achieve this, I would greatly appreciate it. Thanks!

[ghost]

pip install aspose-3d

from argparse import ArgumentParser
from aspose.threed import Scene, FileFormat
from aspose.threed.formats import GltfSaveOptions
import os

parser = ArgumentParser()
parser.add_argument('--input', type=str, required=True, help='path to the input file')
parser.add_argument('--output', type=str, required=True, help='path to the output GLB file')
args = parser.parse_args()

os.chdir(os.path.dirname(args.input))

opt = GltfSaveOptions(FileFormat.GLTF2_BINARY)
opt.embed_assets = True

scn = Scene.from_file(args.input)
scn.save(args.output, opt)

[anasel01]

Thank you so much @4CJ7T for your response. It worked!