mli-resources icon indicating copy to clipboard operation
mli-resources copied to clipboard Machine Learning Interpretability Resources

Machine Learning Interpretability (MLI)

Machine learning algorithms create potentially more accurate models than linear models, but any increase in accuracy over more traditional, better-understood, and more easily explainable techniques is not practical for those who must explain their models to regulators or customers. For many decades, the models created by machine learning algorithms were generally taken to be black-boxes. However, a recent flurry of research has introduced credible techniques for interpreting complex, machine-learned models. Materials presented here illustrate applications or adaptations of these techniques for practicing data scientists.

Want to contribute your own content? Just make a pull request.

Want to use the content in this repo? Just cite the machine learning interpretability team or the original author(s) as appropriate.


Practical MLI examples

(A Dockerfile is provided that will construct a container with all necessary dependencies to run the examples here.)

  • Decision tree surrogate models
  • LIME (practical samples variant)
  • LOCO (NA variant)
  • Partial dependence and individual conditional expectation (ICE)
  • Sensitivity analysis
  • Monotonic models with XGBoost
  • Diabetes data set use case (Diabetes use case has different Dockerfile in seperate repo.)

Installation of Examples


A Dockerfile is provided to build a docker container with all necessary packages and dependencies. This is the easiest way to use these examples if you are on Mac OS X, *nix, or Windows 10. To do so:

  1. Install and start docker. From a terminal:
  2. Create a directory for the Dockerfile. $ mkdir anaconda_py36_h2o_xgboost_graphviz
  3. Fetch the Dockerfile from the mli-resources repo. $ curl > anaconda_py36_h2o_xgboost_graphviz/Dockerfile
  4. Build a docker image from the Dockefile. For this and other docker commands below, you may need to use sudo. $ docker build --no-cache anaconda_py36_h2o_xgboost_graphviz
  5. Display docker image IDs. You are probably interested in the most recently created image. $ docker images
  6. Start the docker image and the Jupyter notebook server. $ docker run -i -t -p 8888:8888 <image_id> /bin/bash -c "/opt/conda/bin/conda install jupyter -y --quiet && /opt/conda/bin/jupyter notebook --notebook-dir=/mli-resources --ip='*' --port=8888 --no-browser --allow-root"
  7. List docker containers. $ docker ps
  8. Copy the sample data into the Docker container. Refer to to obtain datasets needed for notebooks. $ docker cp path/to/train.csv <container_id>:/mli-resources/data/train.csv
  9. Navigate to the port Jupyter directs you to on your machine. It will likely include a token.


  1. Anaconda Python 5.1.0 from the Anaconda archives.
  2. Java.
  3. The latest stable h2o Python package.
  4. Git.
  5. XGBoost with Python bindings.
  6. GraphViz.

Anaconda Python, Java, Git, and GraphViz must be added to your system path.

From a terminal:

  1. Clone the mli-resources repository with examples. $ git clone
  2. $ cd mli-resources
  3. Copy the sample data into the mli-resources repo directory. Refer to to obtain datasets needed for notebooks. $ cp path/to/train.csv ./data
  4. Start the Jupyter notebook server. $ jupyter notebook
  5. Navigate to the port Jupyter directs you to on your machine.

Additional Code Examples

The notebooks in this repo have been revamped and refined many times. Other versions with different, and potentially interesting, details are available at these locations:

Testing Explanations

One way to test generated explanations for accuracy is with simulated data with known characteristics. For instance, models trained on totally random data with no relationship between a number of input variables and a prediction target should not give strong weight to any input variable nor generate compelling local explanations or reason codes. Conversely, you can use simulated data with a known signal generating function to test that explanations accurately represent that known function. Detailed examples of testing explanations with simulated data are available here. A summary of these results are available here.



Conference Presentations

Miscellaneous Resources

General References