✨[Feature] Add a no-conversion flow to Torch-TensorRT

[mfeliz-cruise]

Add a "no_conversion" option to torch-tensorrt which when enabled will replace the standard conversion and engine insertion with an embedded function call for each convertible segment. This allows inspection of the partition without running conversion and the possibility to convert each engine individually in subsequent runs. This could eventually run without requiring a GPU dependency to allow for partition/convertibility checks on host machines without access to a GPU.

Is your feature request related to a problem? Please describe.

Describe the solution you'd like Prototype implementation: https://github.com/pytorch/TensorRT/pull/1360

Describe alternatives you've considered

Additional context

[narendasan]

I am not sure I entirely follow what is produced when this option is enabled. Can you provide a graph or something and some example use cases?

[narendasan]

It sounds to me almost like a dry run feature

[mfeliz-cruise]

Thanks @narendasan I think a "dry run" is a good way of describing it. I've included the output from the ResNet50 unit test in the PR below (aten::add forced to fallback).

Use cases:

1. Run output to check parity with original model, validates lowering.
2. Partition without conversion. Individually convert engine functions to isolate parity issues to a specific engine.
3. Partition without conversion. Individually convert engine functions using different options/settings.
4. Partition without conversion. Lightweight flow to allow quick turnaround while modifying model for convertibility/partitioning.
5. (Proposed) Run torch-tensorrt without GPU access to validate convertibility/inspect partitioning while modifying model.
6. (Proposed) Run torch-tensorrt partitioning without shape propagation. Allows external or manual specification of input shapes at engine boundaries to avoid limitations of existing shape propagation.

module __torch__.torchvision.models.resnet.___torch_mangle_5118.ResNet_trt {
  parameters {
  }
  attributes {
  }
  methods {
    method trt_engine_0x7f5889c90000 {
      graph(%self_1 : __torch__.torchvision.models.resnet.___torch_mangle_5118.ResNet_trt,
            %x_2 : Tensor):
        %self.conv1.weight.1 : Float(64, 3, 7, 7, strides=[147, 49, 7, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %3 : NoneType = prim::Constant() # :0:0
        %4 : int[] = prim::Constant[value=[2, 2]]()
        %5 : int[] = prim::Constant[value=[3, 3]]()
        %6 : int[] = prim::Constant[value=[1, 1]]()
        %7 : bool = prim::Constant[value=0]() # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %8 : int[] = prim::Constant[value=[0, 0]]()
        %9 : int = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:277:0
        %10 : bool = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %self.bn1.weight.1 : Float(64, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.bn1.bias.1 : Float(64, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.bn1.running_mean.1 : Float(64, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.bn1.running_var.1 : Float(64, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %16 : float = prim::Constant[value=0.10000000000000001]() # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %17 : float = prim::Constant[value=1.0000000000000001e-05]() # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %self.layer1.0.conv1.weight.1 : Float(64, 64, 1, 1, strides=[64, 1, 1, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer1.0.bn1.weight.1 : Float(64, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer1.0.bn1.bias.1 : Float(64, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer1.0.bn1.running_mean.1 : Float(64, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer1.0.bn1.running_var.1 : Float(64, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer1.0.conv2.weight.1 : Float(64, 64, 3, 3, strides=[576, 9, 3, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer1.0.bn2.weight.1 : Float(64, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer1.0.bn2.bias.1 : Float(64, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer1.0.bn2.running_mean.1 : Float(64, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer1.0.bn2.running_var.1 : Float(64, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer1.0.conv3.weight.1 : Float(256, 64, 1, 1, strides=[64, 1, 1, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer1.0.downsample.0.weight.1 : Float(256, 64, 1, 1, strides=[64, 1, 1, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer1.0.downsample.1.weight.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer1.0.bn3.bias.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer1.0.downsample.1.running_mean.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer1.0.downsample.1.running_var.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer1.0.bn3.weight.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer1.0.bn3.running_mean.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer1.0.bn3.running_var.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %input.14 : Tensor = aten::_convolution(%x_2, %self.conv1.weight.1, %3, %4, %5, %6, %7, %8, %9, %7, %7, %10, %10) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %input.16 : Tensor = aten::batch_norm(%input.14, %self.bn1.weight.1, %self.bn1.bias.1, %self.bn1.running_mean.1, %self.bn1.running_var.1, %7, %16, %17, %10) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %18 : Tensor = aten::relu(%input.16) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.18 : Tensor = aten::max_pool2d(%18, %5, %4, %6, %6, %7) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:798:0
        %input.22 : Tensor = aten::_convolution(%input.18, %self.layer1.0.conv1.weight.1, %3, %6, %8, %6, %7, %8, %9, %7, %7, %10, %10) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %input.24 : Tensor = aten::batch_norm(%input.22, %self.layer1.0.bn1.weight.1, %self.layer1.0.bn1.bias.1, %self.layer1.0.bn1.running_mean.1, %self.layer1.0.bn1.running_var.1, %7, %16, %17, %10) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %27 : Tensor = aten::relu(%input.24) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.26 : Tensor = aten::_convolution(%27, %self.layer1.0.conv2.weight.1, %3, %6, %6, %6, %7, %8, %9, %7, %7, %10, %10) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %input.28 : Tensor = aten::batch_norm(%input.26, %self.layer1.0.bn2.weight.1, %self.layer1.0.bn2.bias.1, %self.layer1.0.bn2.running_mean.1, %self.layer1.0.bn2.running_var.1, %7, %16, %17, %10) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %35 : Tensor = aten::relu(%input.28) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.30 : Tensor = aten::_convolution(%35, %self.layer1.0.conv3.weight.1, %3, %6, %8, %6, %7, %8, %9, %7, %7, %10, %10) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %input.32 : Tensor = aten::_convolution(%input.18, %self.layer1.0.downsample.0.weight.1, %3, %6, %8, %6, %7, %8, %9, %7, %7, %10, %10) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %identity.2 : Tensor = aten::batch_norm(%input.32, %self.layer1.0.downsample.1.weight.1, %self.layer1.0.bn3.bias.1, %self.layer1.0.downsample.1.running_mean.1, %self.layer1.0.downsample.1.running_var.1, %7, %16, %17, %10) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %out.3 : Tensor = aten::batch_norm(%input.30, %self.layer1.0.bn3.weight.1, %self.layer1.0.bn3.bias.1, %self.layer1.0.bn3.running_mean.1, %self.layer1.0.bn3.running_var.1, %7, %16, %17, %10) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        return (%identity.2, %out.3)
  
    }
    method trt_engine_0x7f5889c901b0 {
      graph(%self_1 : __torch__.torchvision.models.resnet.___torch_mangle_5118.ResNet_trt,
            %1 : Tensor):
        %self.layer1.1.conv1.weight.1 : Float(64, 256, 1, 1, strides=[256, 1, 1, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %4 : NoneType = prim::Constant() # :0:0
        %5 : int[] = prim::Constant[value=[1, 1]]()
        %6 : int[] = prim::Constant[value=[0, 0]]()
        %7 : bool = prim::Constant[value=0]() # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %8 : int = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:277:0
        %9 : bool = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %self.layer1.1.bn1.weight.1 : Float(64, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer1.1.bn1.bias.1 : Float(64, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer1.1.bn1.running_mean.1 : Float(64, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer1.1.bn1.running_var.1 : Float(64, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %15 : float = prim::Constant[value=0.10000000000000001]() # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %16 : float = prim::Constant[value=1.0000000000000001e-05]() # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %self.layer1.1.conv2.weight.1 : Float(64, 64, 3, 3, strides=[576, 9, 3, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer1.1.bn2.weight.1 : Float(64, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer1.1.bn2.bias.1 : Float(64, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer1.1.bn2.running_mean.1 : Float(64, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer1.1.bn2.running_var.1 : Float(64, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer1.1.conv3.weight.1 : Float(256, 64, 1, 1, strides=[64, 1, 1, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer1.1.bn3.weight.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer1.1.bn3.bias.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer1.1.bn3.running_mean.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer1.1.bn3.running_var.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %0 : Tensor = aten::relu(%1) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.36 : Tensor = aten::_convolution(%0, %self.layer1.1.conv1.weight.1, %4, %5, %6, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %input.38 : Tensor = aten::batch_norm(%input.36, %self.layer1.1.bn1.weight.1, %self.layer1.1.bn1.bias.1, %self.layer1.1.bn1.running_mean.1, %self.layer1.1.bn1.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %17 : Tensor = aten::relu(%input.38) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.40 : Tensor = aten::_convolution(%17, %self.layer1.1.conv2.weight.1, %4, %5, %5, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %input.42 : Tensor = aten::batch_norm(%input.40, %self.layer1.1.bn2.weight.1, %self.layer1.1.bn2.bias.1, %self.layer1.1.bn2.running_mean.1, %self.layer1.1.bn2.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %25 : Tensor = aten::relu(%input.42) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.44 : Tensor = aten::_convolution(%25, %self.layer1.1.conv3.weight.1, %4, %5, %6, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %out.5 : Tensor = aten::batch_norm(%input.44, %self.layer1.1.bn3.weight.1, %self.layer1.1.bn3.bias.1, %self.layer1.1.bn3.running_mean.1, %self.layer1.1.bn3.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        return (%0, %out.5)
  
    }
    method trt_engine_0x7f5889c90360 {
      graph(%self_1 : __torch__.torchvision.models.resnet.___torch_mangle_5118.ResNet_trt,
            %1 : Tensor):
        %self.layer1.2.conv1.weight.1 : Float(64, 256, 1, 1, strides=[256, 1, 1, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %4 : NoneType = prim::Constant() # :0:0
        %5 : int[] = prim::Constant[value=[1, 1]]()
        %6 : int[] = prim::Constant[value=[0, 0]]()
        %7 : bool = prim::Constant[value=0]() # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %8 : int = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:277:0
        %9 : bool = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %self.layer1.2.bn1.weight.1 : Float(64, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer1.2.bn1.bias.1 : Float(64, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer1.2.bn1.running_mean.1 : Float(64, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer1.2.bn1.running_var.1 : Float(64, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %15 : float = prim::Constant[value=0.10000000000000001]() # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %16 : float = prim::Constant[value=1.0000000000000001e-05]() # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %self.layer1.2.conv2.weight.1 : Float(64, 64, 3, 3, strides=[576, 9, 3, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer1.2.bn2.weight.1 : Float(64, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer1.2.bn2.bias.1 : Float(64, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer1.2.bn2.running_mean.1 : Float(64, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer1.2.bn2.running_var.1 : Float(64, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer1.2.conv3.weight.1 : Float(256, 64, 1, 1, strides=[64, 1, 1, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer1.2.bn3.weight.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer1.2.bn3.bias.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer1.2.bn3.running_mean.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer1.2.bn3.running_var.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %0 : Tensor = aten::relu(%1) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.48 : Tensor = aten::_convolution(%0, %self.layer1.2.conv1.weight.1, %4, %5, %6, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %input.50 : Tensor = aten::batch_norm(%input.48, %self.layer1.2.bn1.weight.1, %self.layer1.2.bn1.bias.1, %self.layer1.2.bn1.running_mean.1, %self.layer1.2.bn1.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %17 : Tensor = aten::relu(%input.50) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.52 : Tensor = aten::_convolution(%17, %self.layer1.2.conv2.weight.1, %4, %5, %5, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %input.54 : Tensor = aten::batch_norm(%input.52, %self.layer1.2.bn2.weight.1, %self.layer1.2.bn2.bias.1, %self.layer1.2.bn2.running_mean.1, %self.layer1.2.bn2.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %25 : Tensor = aten::relu(%input.54) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.56 : Tensor = aten::_convolution(%25, %self.layer1.2.conv3.weight.1, %4, %5, %6, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %out.7 : Tensor = aten::batch_norm(%input.56, %self.layer1.2.bn3.weight.1, %self.layer1.2.bn3.bias.1, %self.layer1.2.bn3.running_mean.1, %self.layer1.2.bn3.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        return (%0, %out.7)
  
    }
    method trt_engine_0x7f5889c90510 {
      graph(%self_1 : __torch__.torchvision.models.resnet.___torch_mangle_5118.ResNet_trt,
            %1 : Tensor):
        %self.layer2.0.conv1.weight.1 : Float(128, 256, 1, 1, strides=[256, 1, 1, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %4 : NoneType = prim::Constant() # :0:0
        %5 : int[] = prim::Constant[value=[1, 1]]()
        %6 : int[] = prim::Constant[value=[0, 0]]()
        %7 : bool = prim::Constant[value=0]() # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %8 : int = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:277:0
        %9 : bool = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %self.layer2.0.bn1.weight.1 : Float(128, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.0.bn1.bias.1 : Float(128, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.0.bn1.running_mean.1 : Float(128, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.0.bn1.running_var.1 : Float(128, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %15 : float = prim::Constant[value=0.10000000000000001]() # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %16 : float = prim::Constant[value=1.0000000000000001e-05]() # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %self.layer2.0.conv2.weight.1 : Float(128, 128, 3, 3, strides=[1152, 9, 3, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %20 : int[] = prim::Constant[value=[2, 2]]()
        %self.layer2.0.bn2.weight.1 : Float(128, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.0.bn2.bias.1 : Float(128, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.0.bn2.running_mean.1 : Float(128, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.0.bn2.running_var.1 : Float(128, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.0.conv3.weight.1 : Float(512, 128, 1, 1, strides=[128, 1, 1, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.0.downsample.0.weight.1 : Float(512, 256, 1, 1, strides=[256, 1, 1, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.0.downsample.1.weight.1 : Float(512, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.0.bn3.bias.1 : Float(512, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.0.downsample.1.running_mean.1 : Float(512, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.0.downsample.1.running_var.1 : Float(512, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.0.bn3.weight.1 : Float(512, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.0.bn3.running_mean.1 : Float(512, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.0.bn3.running_var.1 : Float(512, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %0 : Tensor = aten::relu(%1) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.60 : Tensor = aten::_convolution(%0, %self.layer2.0.conv1.weight.1, %4, %5, %6, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %input.62 : Tensor = aten::batch_norm(%input.60, %self.layer2.0.bn1.weight.1, %self.layer2.0.bn1.bias.1, %self.layer2.0.bn1.running_mean.1, %self.layer2.0.bn1.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %17 : Tensor = aten::relu(%input.62) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.64 : Tensor = aten::_convolution(%17, %self.layer2.0.conv2.weight.1, %4, %20, %5, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %input.66 : Tensor = aten::batch_norm(%input.64, %self.layer2.0.bn2.weight.1, %self.layer2.0.bn2.bias.1, %self.layer2.0.bn2.running_mean.1, %self.layer2.0.bn2.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %26 : Tensor = aten::relu(%input.66) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.68 : Tensor = aten::_convolution(%26, %self.layer2.0.conv3.weight.1, %4, %5, %6, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %input.70 : Tensor = aten::_convolution(%0, %self.layer2.0.downsample.0.weight.1, %4, %20, %6, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %identity.4 : Tensor = aten::batch_norm(%input.70, %self.layer2.0.downsample.1.weight.1, %self.layer2.0.bn3.bias.1, %self.layer2.0.downsample.1.running_mean.1, %self.layer2.0.downsample.1.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %out.9 : Tensor = aten::batch_norm(%input.68, %self.layer2.0.bn3.weight.1, %self.layer2.0.bn3.bias.1, %self.layer2.0.bn3.running_mean.1, %self.layer2.0.bn3.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        return (%identity.4, %out.9)
  
    }
    method trt_engine_0x7f5889c906c0 {
      graph(%self_1 : __torch__.torchvision.models.resnet.___torch_mangle_5118.ResNet_trt,
            %1 : Tensor):
        %self.layer2.1.conv1.weight.1 : Float(128, 512, 1, 1, strides=[512, 1, 1, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %4 : NoneType = prim::Constant() # :0:0
        %5 : int[] = prim::Constant[value=[1, 1]]()
        %6 : int[] = prim::Constant[value=[0, 0]]()
        %7 : bool = prim::Constant[value=0]() # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %8 : int = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:277:0
        %9 : bool = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %self.layer2.1.bn1.weight.1 : Float(128, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.1.bn1.bias.1 : Float(128, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.1.bn1.running_mean.1 : Float(128, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.1.bn1.running_var.1 : Float(128, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %15 : float = prim::Constant[value=0.10000000000000001]() # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %16 : float = prim::Constant[value=1.0000000000000001e-05]() # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %self.layer2.1.conv2.weight.1 : Float(128, 128, 3, 3, strides=[1152, 9, 3, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.1.bn2.weight.1 : Float(128, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.1.bn2.bias.1 : Float(128, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.1.bn2.running_mean.1 : Float(128, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.1.bn2.running_var.1 : Float(128, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.1.conv3.weight.1 : Float(512, 128, 1, 1, strides=[128, 1, 1, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.1.bn3.weight.1 : Float(512, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.1.bn3.bias.1 : Float(512, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.1.bn3.running_mean.1 : Float(512, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.1.bn3.running_var.1 : Float(512, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %0 : Tensor = aten::relu(%1) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.74 : Tensor = aten::_convolution(%0, %self.layer2.1.conv1.weight.1, %4, %5, %6, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %input.76 : Tensor = aten::batch_norm(%input.74, %self.layer2.1.bn1.weight.1, %self.layer2.1.bn1.bias.1, %self.layer2.1.bn1.running_mean.1, %self.layer2.1.bn1.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %17 : Tensor = aten::relu(%input.76) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.78 : Tensor = aten::_convolution(%17, %self.layer2.1.conv2.weight.1, %4, %5, %5, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %input.80 : Tensor = aten::batch_norm(%input.78, %self.layer2.1.bn2.weight.1, %self.layer2.1.bn2.bias.1, %self.layer2.1.bn2.running_mean.1, %self.layer2.1.bn2.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %25 : Tensor = aten::relu(%input.80) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.82 : Tensor = aten::_convolution(%25, %self.layer2.1.conv3.weight.1, %4, %5, %6, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %out.11 : Tensor = aten::batch_norm(%input.82, %self.layer2.1.bn3.weight.1, %self.layer2.1.bn3.bias.1, %self.layer2.1.bn3.running_mean.1, %self.layer2.1.bn3.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        return (%0, %out.11)
  
    }
    method trt_engine_0x7f5889c90870 {
      graph(%self_1 : __torch__.torchvision.models.resnet.___torch_mangle_5118.ResNet_trt,
            %1 : Tensor):
        %self.layer2.2.conv1.weight.1 : Float(128, 512, 1, 1, strides=[512, 1, 1, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %4 : NoneType = prim::Constant() # :0:0
        %5 : int[] = prim::Constant[value=[1, 1]]()
        %6 : int[] = prim::Constant[value=[0, 0]]()
        %7 : bool = prim::Constant[value=0]() # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %8 : int = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:277:0
        %9 : bool = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %self.layer2.2.bn1.weight.1 : Float(128, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.2.bn1.bias.1 : Float(128, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.2.bn1.running_mean.1 : Float(128, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.2.bn1.running_var.1 : Float(128, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %15 : float = prim::Constant[value=0.10000000000000001]() # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %16 : float = prim::Constant[value=1.0000000000000001e-05]() # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %self.layer2.2.conv2.weight.1 : Float(128, 128, 3, 3, strides=[1152, 9, 3, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.2.bn2.weight.1 : Float(128, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.2.bn2.bias.1 : Float(128, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.2.bn2.running_mean.1 : Float(128, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.2.bn2.running_var.1 : Float(128, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.2.conv3.weight.1 : Float(512, 128, 1, 1, strides=[128, 1, 1, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.2.bn3.weight.1 : Float(512, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.2.bn3.bias.1 : Float(512, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.2.bn3.running_mean.1 : Float(512, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.2.bn3.running_var.1 : Float(512, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %0 : Tensor = aten::relu(%1) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.86 : Tensor = aten::_convolution(%0, %self.layer2.2.conv1.weight.1, %4, %5, %6, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %input.88 : Tensor = aten::batch_norm(%input.86, %self.layer2.2.bn1.weight.1, %self.layer2.2.bn1.bias.1, %self.layer2.2.bn1.running_mean.1, %self.layer2.2.bn1.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %17 : Tensor = aten::relu(%input.88) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.90 : Tensor = aten::_convolution(%17, %self.layer2.2.conv2.weight.1, %4, %5, %5, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %input.92 : Tensor = aten::batch_norm(%input.90, %self.layer2.2.bn2.weight.1, %self.layer2.2.bn2.bias.1, %self.layer2.2.bn2.running_mean.1, %self.layer2.2.bn2.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %25 : Tensor = aten::relu(%input.92) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.94 : Tensor = aten::_convolution(%25, %self.layer2.2.conv3.weight.1, %4, %5, %6, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %out.13 : Tensor = aten::batch_norm(%input.94, %self.layer2.2.bn3.weight.1, %self.layer2.2.bn3.bias.1, %self.layer2.2.bn3.running_mean.1, %self.layer2.2.bn3.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        return (%0, %out.13)
  
    }
    method trt_engine_0x7f5889c90a20 {
      graph(%self_1 : __torch__.torchvision.models.resnet.___torch_mangle_5118.ResNet_trt,
            %1 : Tensor):
        %self.layer2.3.conv1.weight.1 : Float(128, 512, 1, 1, strides=[512, 1, 1, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %4 : NoneType = prim::Constant() # :0:0
        %5 : int[] = prim::Constant[value=[1, 1]]()
        %6 : int[] = prim::Constant[value=[0, 0]]()
        %7 : bool = prim::Constant[value=0]() # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %8 : int = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:277:0
        %9 : bool = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %self.layer2.3.bn1.weight.1 : Float(128, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.3.bn1.bias.1 : Float(128, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.3.bn1.running_mean.1 : Float(128, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.3.bn1.running_var.1 : Float(128, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %15 : float = prim::Constant[value=0.10000000000000001]() # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %16 : float = prim::Constant[value=1.0000000000000001e-05]() # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %self.layer2.3.conv2.weight.1 : Float(128, 128, 3, 3, strides=[1152, 9, 3, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.3.bn2.weight.1 : Float(128, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.3.bn2.bias.1 : Float(128, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.3.bn2.running_mean.1 : Float(128, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.3.bn2.running_var.1 : Float(128, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.3.conv3.weight.1 : Float(512, 128, 1, 1, strides=[128, 1, 1, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.3.bn3.weight.1 : Float(512, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.3.bn3.bias.1 : Float(512, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.3.bn3.running_mean.1 : Float(512, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer2.3.bn3.running_var.1 : Float(512, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %0 : Tensor = aten::relu(%1) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.98 : Tensor = aten::_convolution(%0, %self.layer2.3.conv1.weight.1, %4, %5, %6, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %input.100 : Tensor = aten::batch_norm(%input.98, %self.layer2.3.bn1.weight.1, %self.layer2.3.bn1.bias.1, %self.layer2.3.bn1.running_mean.1, %self.layer2.3.bn1.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %17 : Tensor = aten::relu(%input.100) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.102 : Tensor = aten::_convolution(%17, %self.layer2.3.conv2.weight.1, %4, %5, %5, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %input.104 : Tensor = aten::batch_norm(%input.102, %self.layer2.3.bn2.weight.1, %self.layer2.3.bn2.bias.1, %self.layer2.3.bn2.running_mean.1, %self.layer2.3.bn2.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %25 : Tensor = aten::relu(%input.104) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.106 : Tensor = aten::_convolution(%25, %self.layer2.3.conv3.weight.1, %4, %5, %6, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %out.15 : Tensor = aten::batch_norm(%input.106, %self.layer2.3.bn3.weight.1, %self.layer2.3.bn3.bias.1, %self.layer2.3.bn3.running_mean.1, %self.layer2.3.bn3.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        return (%0, %out.15)
  
    }
    method trt_engine_0x7f5889c90bd0 {
      graph(%self_1 : __torch__.torchvision.models.resnet.___torch_mangle_5118.ResNet_trt,
            %1 : Tensor):
        %self.layer3.0.conv1.weight.1 : Float(256, 512, 1, 1, strides=[512, 1, 1, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %4 : NoneType = prim::Constant() # :0:0
        %5 : int[] = prim::Constant[value=[1, 1]]()
        %6 : int[] = prim::Constant[value=[0, 0]]()
        %7 : bool = prim::Constant[value=0]() # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %8 : int = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:277:0
        %9 : bool = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %self.layer3.0.bn1.weight.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.0.bn1.bias.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.0.bn1.running_mean.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.0.bn1.running_var.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %15 : float = prim::Constant[value=0.10000000000000001]() # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %16 : float = prim::Constant[value=1.0000000000000001e-05]() # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %self.layer3.0.conv2.weight.1 : Float(256, 256, 3, 3, strides=[2304, 9, 3, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %20 : int[] = prim::Constant[value=[2, 2]]()
        %self.layer3.0.bn2.weight.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.0.bn2.bias.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.0.bn2.running_mean.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.0.bn2.running_var.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.0.conv3.weight.1 : Float(1024, 256, 1, 1, strides=[256, 1, 1, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.0.downsample.0.weight.1 : Float(1024, 512, 1, 1, strides=[512, 1, 1, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.0.downsample.1.weight.1 : Float(1024, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.0.bn3.bias.1 : Float(1024, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.0.downsample.1.running_mean.1 : Float(1024, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.0.downsample.1.running_var.1 : Float(1024, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.0.bn3.weight.1 : Float(1024, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.0.bn3.running_mean.1 : Float(1024, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.0.bn3.running_var.1 : Float(1024, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %0 : Tensor = aten::relu(%1) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.110 : Tensor = aten::_convolution(%0, %self.layer3.0.conv1.weight.1, %4, %5, %6, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %input.112 : Tensor = aten::batch_norm(%input.110, %self.layer3.0.bn1.weight.1, %self.layer3.0.bn1.bias.1, %self.layer3.0.bn1.running_mean.1, %self.layer3.0.bn1.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %17 : Tensor = aten::relu(%input.112) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.114 : Tensor = aten::_convolution(%17, %self.layer3.0.conv2.weight.1, %4, %20, %5, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %input.116 : Tensor = aten::batch_norm(%input.114, %self.layer3.0.bn2.weight.1, %self.layer3.0.bn2.bias.1, %self.layer3.0.bn2.running_mean.1, %self.layer3.0.bn2.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %26 : Tensor = aten::relu(%input.116) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.118 : Tensor = aten::_convolution(%26, %self.layer3.0.conv3.weight.1, %4, %5, %6, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %input.120 : Tensor = aten::_convolution(%0, %self.layer3.0.downsample.0.weight.1, %4, %20, %6, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %identity.6 : Tensor = aten::batch_norm(%input.120, %self.layer3.0.downsample.1.weight.1, %self.layer3.0.bn3.bias.1, %self.layer3.0.downsample.1.running_mean.1, %self.layer3.0.downsample.1.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %out.17 : Tensor = aten::batch_norm(%input.118, %self.layer3.0.bn3.weight.1, %self.layer3.0.bn3.bias.1, %self.layer3.0.bn3.running_mean.1, %self.layer3.0.bn3.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        return (%identity.6, %out.17)
  
    }
    method trt_engine_0x7f5889c90d80 {
      graph(%self_1 : __torch__.torchvision.models.resnet.___torch_mangle_5118.ResNet_trt,
            %1 : Tensor):
        %self.layer3.1.conv1.weight.1 : Float(256, 1024, 1, 1, strides=[1024, 1, 1, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %4 : NoneType = prim::Constant() # :0:0
        %5 : int[] = prim::Constant[value=[1, 1]]()
        %6 : int[] = prim::Constant[value=[0, 0]]()
        %7 : bool = prim::Constant[value=0]() # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %8 : int = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:277:0
        %9 : bool = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %self.layer3.1.bn1.weight.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.1.bn1.bias.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.1.bn1.running_mean.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.1.bn1.running_var.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %15 : float = prim::Constant[value=0.10000000000000001]() # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %16 : float = prim::Constant[value=1.0000000000000001e-05]() # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %self.layer3.1.conv2.weight.1 : Float(256, 256, 3, 3, strides=[2304, 9, 3, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.1.bn2.weight.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.1.bn2.bias.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.1.bn2.running_mean.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.1.bn2.running_var.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.1.conv3.weight.1 : Float(1024, 256, 1, 1, strides=[256, 1, 1, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.1.bn3.weight.1 : Float(1024, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.1.bn3.bias.1 : Float(1024, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.1.bn3.running_mean.1 : Float(1024, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.1.bn3.running_var.1 : Float(1024, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %0 : Tensor = aten::relu(%1) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.124 : Tensor = aten::_convolution(%0, %self.layer3.1.conv1.weight.1, %4, %5, %6, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %input.126 : Tensor = aten::batch_norm(%input.124, %self.layer3.1.bn1.weight.1, %self.layer3.1.bn1.bias.1, %self.layer3.1.bn1.running_mean.1, %self.layer3.1.bn1.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %17 : Tensor = aten::relu(%input.126) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.128 : Tensor = aten::_convolution(%17, %self.layer3.1.conv2.weight.1, %4, %5, %5, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %input.130 : Tensor = aten::batch_norm(%input.128, %self.layer3.1.bn2.weight.1, %self.layer3.1.bn2.bias.1, %self.layer3.1.bn2.running_mean.1, %self.layer3.1.bn2.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %25 : Tensor = aten::relu(%input.130) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.132 : Tensor = aten::_convolution(%25, %self.layer3.1.conv3.weight.1, %4, %5, %6, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %out.19 : Tensor = aten::batch_norm(%input.132, %self.layer3.1.bn3.weight.1, %self.layer3.1.bn3.bias.1, %self.layer3.1.bn3.running_mean.1, %self.layer3.1.bn3.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        return (%0, %out.19)
  
    }
    method trt_engine_0x7f5889c90f30 {
      graph(%self_1 : __torch__.torchvision.models.resnet.___torch_mangle_5118.ResNet_trt,
            %1 : Tensor):
        %self.layer3.2.conv1.weight.1 : Float(256, 1024, 1, 1, strides=[1024, 1, 1, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %4 : NoneType = prim::Constant() # :0:0
        %5 : int[] = prim::Constant[value=[1, 1]]()
        %6 : int[] = prim::Constant[value=[0, 0]]()
        %7 : bool = prim::Constant[value=0]() # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %8 : int = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:277:0
        %9 : bool = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %self.layer3.2.bn1.weight.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.2.bn1.bias.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.2.bn1.running_mean.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.2.bn1.running_var.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %15 : float = prim::Constant[value=0.10000000000000001]() # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %16 : float = prim::Constant[value=1.0000000000000001e-05]() # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %self.layer3.2.conv2.weight.1 : Float(256, 256, 3, 3, strides=[2304, 9, 3, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.2.bn2.weight.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.2.bn2.bias.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.2.bn2.running_mean.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.2.bn2.running_var.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.2.conv3.weight.1 : Float(1024, 256, 1, 1, strides=[256, 1, 1, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.2.bn3.weight.1 : Float(1024, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.2.bn3.bias.1 : Float(1024, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.2.bn3.running_mean.1 : Float(1024, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.2.bn3.running_var.1 : Float(1024, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %0 : Tensor = aten::relu(%1) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.29 : Tensor = aten::_convolution(%0, %self.layer3.2.conv1.weight.1, %4, %5, %6, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %input.31 : Tensor = aten::batch_norm(%input.29, %self.layer3.2.bn1.weight.1, %self.layer3.2.bn1.bias.1, %self.layer3.2.bn1.running_mean.1, %self.layer3.2.bn1.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %17 : Tensor = aten::relu(%input.31) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.33 : Tensor = aten::_convolution(%17, %self.layer3.2.conv2.weight.1, %4, %5, %5, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %input.35 : Tensor = aten::batch_norm(%input.33, %self.layer3.2.bn2.weight.1, %self.layer3.2.bn2.bias.1, %self.layer3.2.bn2.running_mean.1, %self.layer3.2.bn2.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %25 : Tensor = aten::relu(%input.35) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.37 : Tensor = aten::_convolution(%25, %self.layer3.2.conv3.weight.1, %4, %5, %6, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %out.6 : Tensor = aten::batch_norm(%input.37, %self.layer3.2.bn3.weight.1, %self.layer3.2.bn3.bias.1, %self.layer3.2.bn3.running_mean.1, %self.layer3.2.bn3.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        return (%0, %out.6)
  
    }
    method trt_engine_0x7f5889c910e0 {
      graph(%self_1 : __torch__.torchvision.models.resnet.___torch_mangle_5118.ResNet_trt,
            %1 : Tensor):
        %self.layer3.3.conv1.weight.1 : Float(256, 1024, 1, 1, strides=[1024, 1, 1, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %4 : NoneType = prim::Constant() # :0:0
        %5 : int[] = prim::Constant[value=[1, 1]]()
        %6 : int[] = prim::Constant[value=[0, 0]]()
        %7 : bool = prim::Constant[value=0]() # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %8 : int = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:277:0
        %9 : bool = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %self.layer3.3.bn1.weight.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.3.bn1.bias.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.3.bn1.running_mean.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.3.bn1.running_var.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %15 : float = prim::Constant[value=0.10000000000000001]() # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %16 : float = prim::Constant[value=1.0000000000000001e-05]() # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %self.layer3.3.conv2.weight.1 : Float(256, 256, 3, 3, strides=[2304, 9, 3, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.3.bn2.weight.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.3.bn2.bias.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.3.bn2.running_mean.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.3.bn2.running_var.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.3.conv3.weight.1 : Float(1024, 256, 1, 1, strides=[256, 1, 1, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.3.bn3.weight.1 : Float(1024, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.3.bn3.bias.1 : Float(1024, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.3.bn3.running_mean.1 : Float(1024, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.3.bn3.running_var.1 : Float(1024, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %0 : Tensor = aten::relu(%1) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.41 : Tensor = aten::_convolution(%0, %self.layer3.3.conv1.weight.1, %4, %5, %6, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %input.43 : Tensor = aten::batch_norm(%input.41, %self.layer3.3.bn1.weight.1, %self.layer3.3.bn1.bias.1, %self.layer3.3.bn1.running_mean.1, %self.layer3.3.bn1.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %17 : Tensor = aten::relu(%input.43) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.45 : Tensor = aten::_convolution(%17, %self.layer3.3.conv2.weight.1, %4, %5, %5, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %input.47 : Tensor = aten::batch_norm(%input.45, %self.layer3.3.bn2.weight.1, %self.layer3.3.bn2.bias.1, %self.layer3.3.bn2.running_mean.1, %self.layer3.3.bn2.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %25 : Tensor = aten::relu(%input.47) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.49 : Tensor = aten::_convolution(%25, %self.layer3.3.conv3.weight.1, %4, %5, %6, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %out.8 : Tensor = aten::batch_norm(%input.49, %self.layer3.3.bn3.weight.1, %self.layer3.3.bn3.bias.1, %self.layer3.3.bn3.running_mean.1, %self.layer3.3.bn3.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        return (%0, %out.8)
  
    }
    method trt_engine_0x7f5889c91290 {
      graph(%self_1 : __torch__.torchvision.models.resnet.___torch_mangle_5118.ResNet_trt,
            %1 : Tensor):
        %self.layer3.4.conv1.weight.1 : Float(256, 1024, 1, 1, strides=[1024, 1, 1, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %4 : NoneType = prim::Constant() # :0:0
        %5 : int[] = prim::Constant[value=[1, 1]]()
        %6 : int[] = prim::Constant[value=[0, 0]]()
        %7 : bool = prim::Constant[value=0]() # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %8 : int = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:277:0
        %9 : bool = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %self.layer3.4.bn1.weight.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.4.bn1.bias.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.4.bn1.running_mean.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.4.bn1.running_var.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %15 : float = prim::Constant[value=0.10000000000000001]() # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %16 : float = prim::Constant[value=1.0000000000000001e-05]() # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %self.layer3.4.conv2.weight.1 : Float(256, 256, 3, 3, strides=[2304, 9, 3, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.4.bn2.weight.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.4.bn2.bias.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.4.bn2.running_mean.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.4.bn2.running_var.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.4.conv3.weight.1 : Float(1024, 256, 1, 1, strides=[256, 1, 1, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.4.bn3.weight.1 : Float(1024, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.4.bn3.bias.1 : Float(1024, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.4.bn3.running_mean.1 : Float(1024, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.4.bn3.running_var.1 : Float(1024, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %0 : Tensor = aten::relu(%1) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.53 : Tensor = aten::_convolution(%0, %self.layer3.4.conv1.weight.1, %4, %5, %6, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %input.55 : Tensor = aten::batch_norm(%input.53, %self.layer3.4.bn1.weight.1, %self.layer3.4.bn1.bias.1, %self.layer3.4.bn1.running_mean.1, %self.layer3.4.bn1.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %17 : Tensor = aten::relu(%input.55) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.57 : Tensor = aten::_convolution(%17, %self.layer3.4.conv2.weight.1, %4, %5, %5, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %input.59 : Tensor = aten::batch_norm(%input.57, %self.layer3.4.bn2.weight.1, %self.layer3.4.bn2.bias.1, %self.layer3.4.bn2.running_mean.1, %self.layer3.4.bn2.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %25 : Tensor = aten::relu(%input.59) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.61 : Tensor = aten::_convolution(%25, %self.layer3.4.conv3.weight.1, %4, %5, %6, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %out.10 : Tensor = aten::batch_norm(%input.61, %self.layer3.4.bn3.weight.1, %self.layer3.4.bn3.bias.1, %self.layer3.4.bn3.running_mean.1, %self.layer3.4.bn3.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        return (%0, %out.10)
  
    }
    method trt_engine_0x7f5889c91440 {
      graph(%self_1 : __torch__.torchvision.models.resnet.___torch_mangle_5118.ResNet_trt,
            %1 : Tensor):
        %self.layer3.5.conv1.weight.1 : Float(256, 1024, 1, 1, strides=[1024, 1, 1, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %4 : NoneType = prim::Constant() # :0:0
        %5 : int[] = prim::Constant[value=[1, 1]]()
        %6 : int[] = prim::Constant[value=[0, 0]]()
        %7 : bool = prim::Constant[value=0]() # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %8 : int = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:277:0
        %9 : bool = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %self.layer3.5.bn1.weight.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.5.bn1.bias.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.5.bn1.running_mean.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.5.bn1.running_var.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %15 : float = prim::Constant[value=0.10000000000000001]() # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %16 : float = prim::Constant[value=1.0000000000000001e-05]() # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %self.layer3.5.conv2.weight.1 : Float(256, 256, 3, 3, strides=[2304, 9, 3, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.5.bn2.weight.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.5.bn2.bias.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.5.bn2.running_mean.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.5.bn2.running_var.1 : Float(256, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.5.conv3.weight.1 : Float(1024, 256, 1, 1, strides=[256, 1, 1, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.5.bn3.weight.1 : Float(1024, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.5.bn3.bias.1 : Float(1024, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.5.bn3.running_mean.1 : Float(1024, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer3.5.bn3.running_var.1 : Float(1024, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %0 : Tensor = aten::relu(%1) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.136 : Tensor = aten::_convolution(%0, %self.layer3.5.conv1.weight.1, %4, %5, %6, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %input.138 : Tensor = aten::batch_norm(%input.136, %self.layer3.5.bn1.weight.1, %self.layer3.5.bn1.bias.1, %self.layer3.5.bn1.running_mean.1, %self.layer3.5.bn1.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %17 : Tensor = aten::relu(%input.138) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.140 : Tensor = aten::_convolution(%17, %self.layer3.5.conv2.weight.1, %4, %5, %5, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %input.142 : Tensor = aten::batch_norm(%input.140, %self.layer3.5.bn2.weight.1, %self.layer3.5.bn2.bias.1, %self.layer3.5.bn2.running_mean.1, %self.layer3.5.bn2.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %25 : Tensor = aten::relu(%input.142) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.144 : Tensor = aten::_convolution(%25, %self.layer3.5.conv3.weight.1, %4, %5, %6, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %out.21 : Tensor = aten::batch_norm(%input.144, %self.layer3.5.bn3.weight.1, %self.layer3.5.bn3.bias.1, %self.layer3.5.bn3.running_mean.1, %self.layer3.5.bn3.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        return (%0, %out.21)
  
    }
    method trt_engine_0x7f5889c915f0 {
      graph(%self_1 : __torch__.torchvision.models.resnet.___torch_mangle_5118.ResNet_trt,
            %1 : Tensor):
        %self.layer4.0.conv1.weight.1 : Float(512, 1024, 1, 1, strides=[1024, 1, 1, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %4 : NoneType = prim::Constant() # :0:0
        %5 : int[] = prim::Constant[value=[1, 1]]()
        %6 : int[] = prim::Constant[value=[0, 0]]()
        %7 : bool = prim::Constant[value=0]() # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %8 : int = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:277:0
        %9 : bool = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %self.layer4.0.bn1.weight.1 : Float(512, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer4.0.bn1.bias.1 : Float(512, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer4.0.bn1.running_mean.1 : Float(512, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer4.0.bn1.running_var.1 : Float(512, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %15 : float = prim::Constant[value=0.10000000000000001]() # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %16 : float = prim::Constant[value=1.0000000000000001e-05]() # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %self.layer4.0.conv2.weight.1 : Float(512, 512, 3, 3, strides=[4608, 9, 3, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %20 : int[] = prim::Constant[value=[2, 2]]()
        %self.layer4.0.bn2.weight.1 : Float(512, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer4.0.bn2.bias.1 : Float(512, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer4.0.bn2.running_mean.1 : Float(512, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer4.0.bn2.running_var.1 : Float(512, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer4.0.conv3.weight.1 : Float(2048, 512, 1, 1, strides=[512, 1, 1, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer4.0.downsample.0.weight.1 : Float(2048, 1024, 1, 1, strides=[1024, 1, 1, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer4.0.downsample.1.weight.1 : Float(2048, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer4.0.bn3.bias.1 : Float(2048, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer4.0.downsample.1.running_mean.1 : Float(2048, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer4.0.downsample.1.running_var.1 : Float(2048, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer4.0.bn3.weight.1 : Float(2048, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer4.0.bn3.running_mean.1 : Float(2048, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer4.0.bn3.running_var.1 : Float(2048, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %0 : Tensor = aten::relu(%1) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.5 : Tensor = aten::_convolution(%0, %self.layer4.0.conv1.weight.1, %4, %5, %6, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %input.7 : Tensor = aten::batch_norm(%input.5, %self.layer4.0.bn1.weight.1, %self.layer4.0.bn1.bias.1, %self.layer4.0.bn1.running_mean.1, %self.layer4.0.bn1.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %17 : Tensor = aten::relu(%input.7) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.9 : Tensor = aten::_convolution(%17, %self.layer4.0.conv2.weight.1, %4, %20, %5, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %input.11 : Tensor = aten::batch_norm(%input.9, %self.layer4.0.bn2.weight.1, %self.layer4.0.bn2.bias.1, %self.layer4.0.bn2.running_mean.1, %self.layer4.0.bn2.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %26 : Tensor = aten::relu(%input.11) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.12 : Tensor = aten::_convolution(%26, %self.layer4.0.conv3.weight.1, %4, %5, %6, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %input.13 : Tensor = aten::_convolution(%0, %self.layer4.0.downsample.0.weight.1, %4, %20, %6, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %identity.1 : Tensor = aten::batch_norm(%input.13, %self.layer4.0.downsample.1.weight.1, %self.layer4.0.bn3.bias.1, %self.layer4.0.downsample.1.running_mean.1, %self.layer4.0.downsample.1.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %out.2 : Tensor = aten::batch_norm(%input.12, %self.layer4.0.bn3.weight.1, %self.layer4.0.bn3.bias.1, %self.layer4.0.bn3.running_mean.1, %self.layer4.0.bn3.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        return (%identity.1, %out.2)
  
    }
    method trt_engine_0x7f5889c917a0 {
      graph(%self_1 : __torch__.torchvision.models.resnet.___torch_mangle_5118.ResNet_trt,
            %1 : Tensor):
        %self.layer4.1.conv1.weight.1 : Float(512, 2048, 1, 1, strides=[2048, 1, 1, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %4 : NoneType = prim::Constant() # :0:0
        %5 : int[] = prim::Constant[value=[1, 1]]()
        %6 : int[] = prim::Constant[value=[0, 0]]()
        %7 : bool = prim::Constant[value=0]() # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %8 : int = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:277:0
        %9 : bool = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %self.layer4.1.bn1.weight.1 : Float(512, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer4.1.bn1.bias.1 : Float(512, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer4.1.bn1.running_mean.1 : Float(512, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer4.1.bn1.running_var.1 : Float(512, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %15 : float = prim::Constant[value=0.10000000000000001]() # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %16 : float = prim::Constant[value=1.0000000000000001e-05]() # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %self.layer4.1.conv2.weight.1 : Float(512, 512, 3, 3, strides=[4608, 9, 3, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer4.1.bn2.weight.1 : Float(512, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer4.1.bn2.bias.1 : Float(512, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer4.1.bn2.running_mean.1 : Float(512, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer4.1.bn2.running_var.1 : Float(512, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer4.1.conv3.weight.1 : Float(2048, 512, 1, 1, strides=[512, 1, 1, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer4.1.bn3.weight.1 : Float(2048, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer4.1.bn3.bias.1 : Float(2048, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer4.1.bn3.running_mean.1 : Float(2048, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer4.1.bn3.running_var.1 : Float(2048, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %0 : Tensor = aten::relu(%1) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.17 : Tensor = aten::_convolution(%0, %self.layer4.1.conv1.weight.1, %4, %5, %6, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %input.19 : Tensor = aten::batch_norm(%input.17, %self.layer4.1.bn1.weight.1, %self.layer4.1.bn1.bias.1, %self.layer4.1.bn1.running_mean.1, %self.layer4.1.bn1.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %17 : Tensor = aten::relu(%input.19) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.21 : Tensor = aten::_convolution(%17, %self.layer4.1.conv2.weight.1, %4, %5, %5, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %input.23 : Tensor = aten::batch_norm(%input.21, %self.layer4.1.bn2.weight.1, %self.layer4.1.bn2.bias.1, %self.layer4.1.bn2.running_mean.1, %self.layer4.1.bn2.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %25 : Tensor = aten::relu(%input.23) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.25 : Tensor = aten::_convolution(%25, %self.layer4.1.conv3.weight.1, %4, %5, %6, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %out.4 : Tensor = aten::batch_norm(%input.25, %self.layer4.1.bn3.weight.1, %self.layer4.1.bn3.bias.1, %self.layer4.1.bn3.running_mean.1, %self.layer4.1.bn3.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        return (%0, %out.4)
  
    }
    method trt_engine_0x7f5889c91950 {
      graph(%self_1 : __torch__.torchvision.models.resnet.___torch_mangle_5118.ResNet_trt,
            %1 : Tensor):
        %self.layer4.2.conv1.weight.1 : Float(512, 2048, 1, 1, strides=[2048, 1, 1, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %4 : NoneType = prim::Constant() # :0:0
        %5 : int[] = prim::Constant[value=[1, 1]]()
        %6 : int[] = prim::Constant[value=[0, 0]]()
        %7 : bool = prim::Constant[value=0]() # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %8 : int = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:277:0
        %9 : bool = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %self.layer4.2.bn1.weight.1 : Float(512, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer4.2.bn1.bias.1 : Float(512, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer4.2.bn1.running_mean.1 : Float(512, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer4.2.bn1.running_var.1 : Float(512, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %15 : float = prim::Constant[value=0.10000000000000001]() # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %16 : float = prim::Constant[value=1.0000000000000001e-05]() # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %self.layer4.2.conv2.weight.1 : Float(512, 512, 3, 3, strides=[4608, 9, 3, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer4.2.bn2.weight.1 : Float(512, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer4.2.bn2.bias.1 : Float(512, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer4.2.bn2.running_mean.1 : Float(512, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer4.2.bn2.running_var.1 : Float(512, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer4.2.conv3.weight.1 : Float(2048, 512, 1, 1, strides=[512, 1, 1, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer4.2.bn3.weight.1 : Float(2048, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer4.2.bn3.bias.1 : Float(2048, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer4.2.bn3.running_mean.1 : Float(2048, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.layer4.2.bn3.running_var.1 : Float(2048, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %0 : Tensor = aten::relu(%1) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.4 : Tensor = aten::_convolution(%0, %self.layer4.2.conv1.weight.1, %4, %5, %6, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %input.6 : Tensor = aten::batch_norm(%input.4, %self.layer4.2.bn1.weight.1, %self.layer4.2.bn1.bias.1, %self.layer4.2.bn1.running_mean.1, %self.layer4.2.bn1.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %17 : Tensor = aten::relu(%input.6) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.8 : Tensor = aten::_convolution(%17, %self.layer4.2.conv2.weight.1, %4, %5, %5, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %input.10 : Tensor = aten::batch_norm(%input.8, %self.layer4.2.bn2.weight.1, %self.layer4.2.bn2.bias.1, %self.layer4.2.bn2.running_mean.1, %self.layer4.2.bn2.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        %25 : Tensor = aten::relu(%input.10) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %input.1 : Tensor = aten::_convolution(%25, %self.layer4.2.conv3.weight.1, %4, %5, %6, %5, %7, %6, %8, %7, %7, %9, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py:443:0
        %out.1 : Tensor = aten::batch_norm(%input.1, %self.layer4.2.bn3.weight.1, %self.layer4.2.bn3.bias.1, %self.layer4.2.bn3.running_mean.1, %self.layer4.2.bn3.running_var.1, %7, %15, %16, %9) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:2411:0
        return (%0, %out.1)
  
    }
    method trt_engine_0x7f5889c91b00 {
      graph(%self_1 : __torch__.torchvision.models.resnet.___torch_mangle_5118.ResNet_trt,
            %1 : Tensor):
        %3 : int[] = prim::Constant[value=[1, 1]]()
        %5 : int = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:277:0
        %6 : int = prim::Constant[value=-1]()
        %self.fc.weight : Float(1000, 2048, strides=[2048, 1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %self.fc.bias : Float(1000, strides=[1], requires_grad=0, device=cuda:0) = prim::Constant[value=<Tensor>]()
        %0 : Tensor = aten::relu(%1) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1408:0
        %x.1 : Tensor = aten::adaptive_avg_pool2d(%0, %3) # /opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:1242:0
        %input.20 : Tensor = aten::flatten(%x.1, %5, %6) # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:277:0
        %7 : Tensor = aten::t(%self.fc.weight)
        %9 : Tensor = aten::matmul(%input.20, %7)
        %10 : Tensor = trt::const(%self.fc.bias)
        return (%9, %10)
  
    }
    method forward {
      graph(%input_0 : __torch__.torchvision.models.resnet.___torch_mangle_5118.ResNet_trt,
            %input_1 : Tensor):
        %2 : Tensor, %3 : Tensor = prim::CallMethod[name="trt_engine_0x7f5889c90000"](%input_0, %input_1)
        %4 : int = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:277:0
        %5 : Tensor = aten::add(%3, %2, %4) # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:158:0
        %6 : Tensor, %7 : Tensor = prim::CallMethod[name="trt_engine_0x7f5889c901b0"](%input_0, %5)
        %8 : int = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:277:0
        %9 : Tensor = aten::add(%7, %6, %8) # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:158:0
        %10 : Tensor, %11 : Tensor = prim::CallMethod[name="trt_engine_0x7f5889c90360"](%input_0, %9)
        %12 : int = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:277:0
        %13 : Tensor = aten::add(%11, %10, %12) # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:158:0
        %14 : Tensor, %15 : Tensor = prim::CallMethod[name="trt_engine_0x7f5889c90510"](%input_0, %13)
        %16 : int = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:277:0
        %17 : Tensor = aten::add(%15, %14, %16) # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:158:0
        %18 : Tensor, %19 : Tensor = prim::CallMethod[name="trt_engine_0x7f5889c906c0"](%input_0, %17)
        %20 : int = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:277:0
        %21 : Tensor = aten::add(%19, %18, %20) # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:158:0
        %22 : Tensor, %23 : Tensor = prim::CallMethod[name="trt_engine_0x7f5889c90870"](%input_0, %21)
        %24 : int = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:277:0
        %25 : Tensor = aten::add(%23, %22, %24) # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:158:0
        %26 : Tensor, %27 : Tensor = prim::CallMethod[name="trt_engine_0x7f5889c90a20"](%input_0, %25)
        %28 : int = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:277:0
        %29 : Tensor = aten::add(%27, %26, %28) # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:158:0
        %30 : Tensor, %31 : Tensor = prim::CallMethod[name="trt_engine_0x7f5889c90bd0"](%input_0, %29)
        %32 : int = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:277:0
        %33 : Tensor = aten::add(%31, %30, %32) # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:158:0
        %34 : Tensor, %35 : Tensor = prim::CallMethod[name="trt_engine_0x7f5889c90d80"](%input_0, %33)
        %36 : int = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:277:0
        %37 : Tensor = aten::add(%35, %34, %36) # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:158:0
        %38 : Tensor, %39 : Tensor = prim::CallMethod[name="trt_engine_0x7f5889c90f30"](%input_0, %37)
        %40 : int = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:277:0
        %41 : Tensor = aten::add(%39, %38, %40) # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:158:0
        %42 : Tensor, %43 : Tensor = prim::CallMethod[name="trt_engine_0x7f5889c910e0"](%input_0, %41)
        %44 : int = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:277:0
        %45 : Tensor = aten::add(%43, %42, %44) # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:158:0
        %46 : Tensor, %47 : Tensor = prim::CallMethod[name="trt_engine_0x7f5889c91290"](%input_0, %45)
        %48 : int = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:277:0
        %49 : Tensor = aten::add(%47, %46, %48) # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:158:0
        %50 : Tensor, %51 : Tensor = prim::CallMethod[name="trt_engine_0x7f5889c91440"](%input_0, %49)
        %52 : int = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:277:0
        %53 : Tensor = aten::add(%51, %50, %52) # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:158:0
        %54 : Tensor, %55 : Tensor = prim::CallMethod[name="trt_engine_0x7f5889c915f0"](%input_0, %53)
        %56 : int = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:277:0
        %57 : Tensor = aten::add(%55, %54, %56) # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:158:0
        %58 : Tensor, %59 : Tensor = prim::CallMethod[name="trt_engine_0x7f5889c917a0"](%input_0, %57)
        %60 : int = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:277:0
        %61 : Tensor = aten::add(%59, %58, %60) # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:158:0
        %62 : Tensor, %63 : Tensor = prim::CallMethod[name="trt_engine_0x7f5889c91950"](%input_0, %61)
        %64 : int = prim::Constant[value=1]() # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:277:0
        %65 : Tensor = aten::add(%63, %62, %64) # /opt/conda/lib/python3.8/site-packages/torchvision/models/resnet.py:158:0
        %66 : Tensor, %67 : Tensor = prim::CallMethod[name="trt_engine_0x7f5889c91b00"](%input_0, %65)
        %68 : int = prim::Constant[value=1]()
        %69 : Tensor = aten::add(%67, %66, %68)
        return (%69)
  
    }
  }
  submodules {
  }
}

[narendasan]

Hmm, I think theres definitely a lot of useful repercussions of a feature like this. Off the top of my head I could see this simplifying some of the core/compiler logic, make collections more standard and easy to handle. It would force us to add support for compiling methods other than forward (#621).

Id probably like formalize the design for this feature in an RFC (similar to https://github.com/pytorch/TensorRT/discussions/629) so we can think through all the parts: UX, implications for other components etc. Like I imagine one thing that would be nice is to have better names for trt_methods if users would interact with them directly.

[mfeliz-cruise]

RFC created here https://github.com/pytorch/TensorRT/discussions/1369

[github-actions[bot]]

This issue has not seen activity for 90 days, Remove stale label or comment or this will be closed in 10 days

[github-actions[bot]]

This issue has not seen activity for 90 days, Remove stale label or comment or this will be closed in 10 days