❓ [Question] How to save the compiled while using torch.compile

[klin2024]

For the example below, how do I save the compiled model?

backend = "torch_tensorrt" tp_model = torch.compile( tp_model, backend=backend, options={ "truncate_long_and_double": True, "enabled_precisions": {torch.float32, torch.float16}, "use_python_runtime": True, "min_block_size": 1, }, dynamic=False, )

[narendasan]

torch.compile is not serializable. There are two options. You can enable the engine cache which create a persistent cache you can use between sessions or you could use the MutableTorchTensorRTModule which has a similar UX but is serializable