serve
serve copied to clipboard
Openvino integration into torchserve
Description
This PR adds initial support for integrating OpenVINO backend for torch.compile into torchserve. With this change users will be able to serve their pytorch models on torchserve and accelerate them using OpenVINO on Intel hardware.
Type of change
- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
- [x] New feature (non-breaking change which adds functionality)
- [x] This change requires a documentation update
Checklist:
- [ ] Did you have fun?
- [ ] Have you added tests that prove your fix is effective or that this feature works?
- [ ] Has code been commented, particularly in hard-to-understand areas?
- [ ] Have you made corresponding changes to the documentation?