to edge IR from transformers library model

[mhs4670go]

Hello. I'm trying to deploy llama model from transformers library on my backend. But, before going to the to_backend api, to_edge api raised an error. I know there is llama2 model in the examples/models/llama2 but it'd be convenient if I can do this.

Am I doing something wrong or there is no plan to support this?

import torch
from torch.export import export
from executorch.exir import to_edge
 
config = LlamaConfig()
# for lightweight test
config.num_hidden_layers = 1
 
m = LlamaForCausalLM(config)
 
with torch.no_grad():
  ep = export(m, (m.dummy_inputs['input_ids'],))

# error
module_edge = to_edge(model)

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/sw4670.chae/-exir/_exir/utils/utils.py", line 14, in convert_exported_module_to_
    module_edge = to_edge(model)
  File "/home/sw4670.chae/.venv/lib/python3.10/site-packages/executorch/exir/program/_program.py", line 631, in to_edge
    program = program.run_decompositions(_default_decomposition_table())
  File "/home/sw4670.chae/.venv/lib/python3.10/site-packages/torch/export/exported_program.py", line 84, in wrapper
    return fn(*args, **kwargs)
  File "/home/sw4670.chae/.venv/lib/python3.10/site-packages/torch/export/exported_program.py", line 480, in run_decompositions
    gm, graph_signature = aot_export_module(
  File "/home/sw4670.chae/.venv/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py", line 1047, in aot_export_module
    fx_g, metadata, in_spec, out_spec = _aot_export_function(
  File "/home/sw4670.chae/.venv/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py", line 1237, in _aot_export_function
    fx_g, meta = create_aot_dispatcher_function(
  File "/home/sw4670.chae/.venv/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 265, in time_wrapper
    r = func(*args, **kwargs)
  File "/home/sw4670.chae/.venv/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py", line 628, in create_aot_dispatcher_function
    compiled_fn = compiler_fn(flat_fn, fake_flat_args, aot_config, fw_metadata=fw_metadata)
  File "/home/sw4670.chae/.venv/lib/python3.10/site-packages/torch/_functorch/_aot_autograd/runtime_wrappers.py", line 444, in aot_wrapper_dedupe
    return compiler_fn(flat_fn, leaf_flat_args, aot_config, fw_metadata=fw_metadata)
  File "/home/sw4670.chae/.venv/lib/python3.10/site-packages/torch/_functorch/_aot_autograd/runtime_wrappers.py", line 649, in aot_wrapper_synthetic_base
    return compiler_fn(flat_fn, flat_args, aot_config, fw_metadata=fw_metadata)
  File "/home/sw4670.chae/.venv/lib/python3.10/site-packages/torch/_functorch/_aot_autograd/dispatch_and_compile_graph.py", line 97, in aot_dispatch_base_graph
    copy_count = assert_functional_graph(fw_module.graph)
  File "/home/sw4670.chae/.venv/lib/python3.10/site-packages/torch/_functorch/_aot_autograd/functional_utils.py", line 327, in assert_functional_graph
    assert n.args[0] in placeholders
AssertionError

[mhs4670go]

Hmm.. I'm not sure but it worked in different torch version. I will close this issue soon.

commit: 643c628

# error (with GPU)
torch                    2.4.0.dev20240324+cu118
torchaudio               2.2.0.dev20240324+cpu
torchsr                  1.0.4
torchvision              0.19.0.dev20240324+cpu

# work (with cpu)
torch                 2.4.0.dev20240324+cpu
torchaudio            2.2.0.dev20240324+cpu
torchsr               1.0.4
torchvision           0.19.0.dev20240324+cpu

[iseeyuan]

@mhs4670go Yes, ExecuTorch is pinned to a specific PyTorch version. You may run ./install_requirements to set it up.