William Hicks

Update: I was able to get all of the necessary changes into the FIL backend itself, and I've prepped the build workflow changes necessary to add it to our ARM...

My apologies folks; this came in just too close to the release window, so it looks like it is likely to land in 23.12. In the meantime, you can certainly...

@guptap11 My apologies; this fell through the cracks while I was on vacation. I just checked and it does _not_ look like the FIL backend made it into the 23.12...

Update: I've set up an environment to manually test and validate the build on Jetson, but I had some issues getting access to the correct board model so I have...

There has been some discussion of it in the past, but it is not currently a high priority on our roadmap. If you can provide more details on what you're...

Thank you for the report! Could you post how the model was generated and the model config file you used to load it into Triton?

Possibly related: https://github.com/dmlc/treelite/issues/364. If that is indeed the underlying issue, the `use_experimental_optimizations` [flag](https://nbviewer.org/github/triton-inference-server/fil_backend/blob/main/notebooks/faq/FAQs.ipynb#FAQ-9:-How-can-we-improve-performance-of-models-deployed-with-the-FIL-backend?) may be a workaround for the moment.

Hmmm... I don't see why that particular model would trigger that Treelite issue, so we may need to dig deeper. Can you try the `use_experimental_optimizations` flag and let me know...

Apologies; I was too hasty when I was thinking about this before. As soon as I saw `LLVM`, I was thinking about Treelite compiled models, but the FIL backend does...

@sandeepb2013 Could you try with an officially-released Triton Docker image and enable `use_experimental_optimizations` in your config.pbtxt? The memory requirements should be quite modest, though they'll depend on the details of...