Andy comments

Results 30 comments of


                                            Andy

TensorRT Initial commit

Actually, I have been experimenting around with the docs stuff in your branch. I can do a quick fix right now.

TensorRT Initial commit

The Sphinx docs generation requires module imports `import tensorrt as trt` and `from cuda import cuda`. I think our script just needs to detect that those modules exist in `tensorrt_inference.py`...

TensorRT Initial commit

Oh, great to know! Thanks @pranavm-nvidia, I'll check that out

TensorRT Initial commit

Good job everyone, and thanks @damccorm for the merge!

[Feature Request]: Allow specification of a custom model inference method for a RunInference ModelHandler

This also applies to scikit-learn. For example, [RandomForestClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html) has `predict(X)`, `predict_proba(X)` or `predict_log_proba(X)`, and other less common functions like `apply(X)`, etc.

[Feature Request]: Allow specification of a custom model inference method for a RunInference ModelHandler

Parent issue: https://github.com/apache/beam/issues/22117

[Feature Request]: Allow specification of a custom model inference method for a RunInference ModelHandler

> pytorch calls the model like a callable (which then uses the forward method IIUC?): Correct. --- And thanks @agvdndor for the detailed suggestions! - For `GenerationModelHandler` I agree that...