Honey, I broke the PyTorch model

🍯😊⛏️🐍🔥🧮

Talk at PyCon.DE & PyData Berlin 2023

Are you ready to lift the veil from your broken PyTorch model and prevent it from ever breaking again? This presentation covers strategies to

1. Create synthetic data for your custom ML model and
2. Setup an adequate test suite to speed up your ML dev process

To run the streamlit app, follow the steps below:

1. Set-up

Activate the poetry environment

poetry shell

and run the presentation

streamlit run Home.py

The presentation should now be available in a new browser tab

2. PyTorch examples

All PyTorch code is contained in pages/torch_examples

3. Sources & further reading

Testing in ML

• Jeremy Jordan's blogpost about testing philosophy for ML
• Testing for PyTorch with torchcheck
• Deprecated, but interesting test suites for < TF2.0: mltest and for PyTorch: torchtest
• Differences between Software Engineering and Machine Learning Engineering workflows at TensorFlow explained by TF team lead

Post-training checks

• Weight analysis without training or validation data: weightwatcher
• Post-training label quality analysis: cleanlab
• Behavioral testing for NLP models

For other bugs and better performance

• Andrej Kaparthy's recipe for training NNs
• and if you're done debugging check out Google's Deep Learning tuning playbook