[Examples] Provide end2end examples using byteir ( fronts compiler and runtime ) ,for example resnet18

[lyffly]

hello , can you provide end2end examples using byteir ( fronts compiler and runtime ) ,for example resnet18

[qingyunqu]

Hi, you could try like https://github.com/bytedance/byteir/tree/main/frontends/torch-frontend/examples/demo. But the demo seems to be a little outdated. If you meet any problem when you try the demo, feel free to ask me.

[recursionstudio]

Hi, you could try like https://github.com/bytedance/byteir/tree/main/frontends/torch-frontend/examples/demo. But the demo seems to be a little outdated. If you meet any problem when you try the demo, feel free to ask me.

Hi I run gpt2 under demo recently and got following errors： /host/byteir/frontends/torch-frontend/examples/demo# python3 main.py --infer gpt2

python3: /host/byteir/frontends/torch-frontend/third_party/torch-mlir/lib/Dialect/Torch/IR/TorchDialect.cpp:176: virtual mlir::Operation* mlir::torch::Torch::Torch Dialect::materializeConstant(mlir::OpBuilder&, mlir::Attribute, mlir::Type, mlir::Location): Assertion `isa<ValueTensorType>(type) && "should be a vtensor type!"' failed. Aborted (core dumped)

/host/byteir/frontends/torch-frontend/examples/demo# python3 main.py gpt2

File "/host/byteir/frontends/torch-frontend/examples/demo/backend.py", line 205, in byteir_runner byteir.compile(mlir_file_name, output_mlir_file_name, entry_func='forward', target='cuda') File "/root/anaconda3/lib/python3.9/site-packages/byteir/compile.py", line 223, in compile module = ir.Module.parse(open(input_file_path, "r").read(), context) byteir._mlir_libs._site_initialize..MLIRError: Unable to parse module assembly: error: "-":134:12: 'stablehlo.gather' op attribute 'slice_sizes' failed to satisfy constraint: 64-bit signless integer elements attribute note: "-":134:12: see current operation: %131 = "stablehlo.gather"(%arg111, %78) {dimension_numbers = #stablehlo.gather<offset_dims = [2], collapsed_slice_dims = [0], start_index_map = [0], index_vector_dim = 2>, indices_are_sorted = false, slice_sizes = array<i64: 1, 768>} : (tensor<50257x768xf32>, tensor<8x1024xi64>) -> t ensor<8x1024x768xf32>

Could you take a look? My commit is f734a1f8af3ea35495a9dce1c58eb922d17b50b8

[lyffly]

1、use https://github.com/bytedance/byteir/blob/main/frontends/torch-frontend/examples/inference/infer_resnet.py generate resnet.stablehlo.mlir 2、then compile to cuda

from byteir import compile
compile("./resnet.stablehlo.mlir", "./out.mlir", target="cuda", verbose=True)

error shows :

compile("./resnet18.stablehlo.mlir", "./resnet18.cuda.mlir", target="cuda", verbose=True)
  File "/usr/local/lib/python3.8/dist-packages/byteir/compile.py", line 212, in compile
    module = ir.Module.parse(open(input_file_path, "r").read(), context)
byteir._mlir_libs.MLIRError: Unable to parse module assembly:
error: "-":113:12: 'stablehlo.reduce_window' op attribute 'window_dimensions' failed to satisfy constraint: 64-bit signless integer elements attribute
 note: "-":113:12: see current operation: 
  %110 = "stablehlo.reduce_window"(%109, %101) ({
  ^bb0(%arg1: tensor<f32>, %arg2: tensor<f32>):
    %183 = "stablehlo.maximum"(%arg1, %arg2) : (tensor<f32>, tensor<f32>) -> tensor<f32>
    "stablehlo.return"(%183) : (tensor<f32>) -> ()
  }) {padding = dense<[[0, 0], [0, 0], [1, 1], [1, 1]]> : tensor<4x2xi64>, window_dilations = array<i64: 1, 1, 1, 1>, window_dimensions = array<i64: 1, 1, 3, 3>, window_strides = array<i64: 1, 1, 2, 2>} : (tensor<2x64x112x112xf32>, tensor<f32>) -> tensor<2x64x56x56xf32>

[qingyunqu]

1、use https://github.com/bytedance/byteir/blob/main/frontends/torch-frontend/examples/inference/infer_resnet.py generate resnet.stablehlo.mlir 2、then compile to cuda

from byteir import compile
compile("./resnet.stablehlo.mlir", "./out.mlir", target="cuda", verbose=True)

error shows :

compile("./resnet18.stablehlo.mlir", "./resnet18.cuda.mlir", target="cuda", verbose=True)
  File "/usr/local/lib/python3.8/dist-packages/byteir/compile.py", line 212, in compile
    module = ir.Module.parse(open(input_file_path, "r").read(), context)
byteir._mlir_libs.MLIRError: Unable to parse module assembly:
error: "-":113:12: 'stablehlo.reduce_window' op attribute 'window_dimensions' failed to satisfy constraint: 64-bit signless integer elements attribute
 note: "-":113:12: see current operation: 
  %110 = "stablehlo.reduce_window"(%109, %101) ({
  ^bb0(%arg1: tensor<f32>, %arg2: tensor<f32>):
    %183 = "stablehlo.maximum"(%arg1, %arg2) : (tensor<f32>, tensor<f32>) -> tensor<f32>
    "stablehlo.return"(%183) : (tensor<f32>) -> ()
  }) {padding = dense<[[0, 0], [0, 0], [1, 1], [1, 1]]> : tensor<4x2xi64>, window_dilations = array<i64: 1, 1, 1, 1>, window_dimensions = array<i64: 1, 1, 3, 3>, window_strides = array<i64: 1, 1, 2, 2>} : (tensor<2x64x112x112xf32>, tensor<f32>) -> tensor<2x64x56x56xf32>

In this case, the root cause is that torch-frontend has different stablehlo version with byteir. You could use stablehlo's bytecode to fill the gap. Here is example code:

import byteir
from byteir._mlir_libs._stablehlo import get_current_version
import torch_frontend

module_bytes = torch_frontend.compile(Resnet18(), sample_inputs, "stablehlo+"+get_current_version())
with open("./resnet.mlirbc", "wb") as f:
  f.write(module_bytes)

byteir.compile("./resnet.mlirbc", "./out.mlir", entry_func="forward", target="cuda", verbose=True)

[lyffly]

1、use https://github.com/bytedance/byteir/blob/main/frontends/torch-frontend/examples/inference/infer_resnet.py generate resnet.stablehlo.mlir 2、then compile to cuda

from byteir import compile
compile("./resnet.stablehlo.mlir", "./out.mlir", target="cuda", verbose=True)

error shows :

compile("./resnet18.stablehlo.mlir", "./resnet18.cuda.mlir", target="cuda", verbose=True)
  File "/usr/local/lib/python3.8/dist-packages/byteir/compile.py", line 212, in compile
    module = ir.Module.parse(open(input_file_path, "r").read(), context)
byteir._mlir_libs.MLIRError: Unable to parse module assembly:
error: "-":113:12: 'stablehlo.reduce_window' op attribute 'window_dimensions' failed to satisfy constraint: 64-bit signless integer elements attribute
 note: "-":113:12: see current operation: 
  %110 = "stablehlo.reduce_window"(%109, %101) ({
  ^bb0(%arg1: tensor<f32>, %arg2: tensor<f32>):
    %183 = "stablehlo.maximum"(%arg1, %arg2) : (tensor<f32>, tensor<f32>) -> tensor<f32>
    "stablehlo.return"(%183) : (tensor<f32>) -> ()
  }) {padding = dense<[[0, 0], [0, 0], [1, 1], [1, 1]]> : tensor<4x2xi64>, window_dilations = array<i64: 1, 1, 1, 1>, window_dimensions = array<i64: 1, 1, 3, 3>, window_strides = array<i64: 1, 1, 2, 2>} : (tensor<2x64x112x112xf32>, tensor<f32>) -> tensor<2x64x56x56xf32>

In this case, the root cause is that torch-frontend has different stablehlo version with byteir. You could use stablehlo's bytecode to fill the gap. Here is example code:

import byteir
from byteir._mlir_libs._stablehlo import get_current_version
import torch_frontend

module_bytes = torch_frontend.compile(Resnet18(), sample_inputs, "stablehlo+"+get_current_version())
with open("./resnet.mlirbc", "wb") as f:
  f.write(module_bytes)

byteir.compile("./resnet.mlirbc", "./out.mlir", entry_func="forward", target="cuda", verbose=True)

Get it. Thanks .

[lyffly]

Close this issue.