Working demo?

[surak]

Hey, I just got here via the TLDR Newsletter (https://tldr.tech)

This project looks great! However, some screenshots or even a demo would be a nice showcase of what's expected to happen on deployment. Currently it's hard to see what or how it would look like over it.

If there is something like this, it's not obvious - I skimmed over the documentation and the readme and couldn't find it.

Thanks!

[jhoetter]

Hi @surak,

not a Baseten member, but I thought this one here was quite straightforward and worked for me:

!pip install scikit-learn
!pip install truss
import truss
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import load_iris

# Load the iris data set
iris = load_iris()
data_x = iris['data']
data_y = iris['target']

# Train the model
rfc = RandomForestClassifier()
rfc.fit(data_x, data_y)

# Create the Truss (serializing & packaging model)
tr = truss.mk_truss(rfc, target_directory="iris_rfc_truss")

# Serve a prediction from the model
tr.server_predict({"inputs": [[0, 0, 0, 0]]})

Then run truss run-image iris_rfc_truss on your terminal, and afterward curl -X POST http://127.0.0.1:8080/v1/models/model:predict -d '{"inputs": [[0, 0, 0, 0]]}'

I think it's quite straightforward. Works nicely for me :)

You can find the demo here: https://truss.baseten.co/

[pankajroark]

Thanks for your request. There should be a demo video very soon, it's in the works. Meanwhile, you could look at the deploy guides in the docs for AWS, GCP, or Baseten. Deploying to local docker should be simple as @jhoetter described above, truss would basically just launch a container on your local docker that you can then hit with HTTP requests.

[philipkiely-baseten]

Our developer advocate Jesse has some fantastic examples on Kaggle:

• https://www.kaggle.com/code/jessemostipak/from-kaggle-to-prod-using-truss-pytorch
• https://www.kaggle.com/code/jessemostipak/truss-hugging-face
• https://www.kaggle.com/code/jessemostipak/truss-xgboost-for-rapid-model-deployment

These notebooks demo training a model, packaging it as a Truss, and deploying it to Baseten, and include screenshots of the expected terminal output during a deployment