Noah Golmant

Incorrect documentation for `warm_start` behavior on BaseForest-derived classes

6

#### Describe the issue linked to the documentation The [RandomForestRegressor](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestRegressor.html) documentation states: > When set to True, reuse the solution of the previous call to fit and add more estimators...

Upload to pypi registry

7

It would be great to upload this project to pypi so that it is more easily installable from other package management tools. Right now, we install our package dependencies using...

Add spectral density computation using lanczos vectors

2

Implement Stochastic Lanczos Quadrature using scipy or a custom Lanczos implementation. Integrates the approach from [this](https://github.com/google/spectral-density) repo.

Add a Colab notebook that measures the Hessian spectrum for a BERT or GPT-style language model.

Add and test multi-GPU support and MPI integration. I need GPUs for this.

Add continuous Hessian spectrum throughout training

3

Add GROUSE implementation for efficient low-rank subspace tracking with Grassmannian SGD based on the following paper (or other more recent subspace tracking papers): https://people.eecs.berkeley.edu/~brecht/papers/10.Bal.Now.Rec.GROUSE.pdf

This PR should: - Provide tracking class to update estimates with incremental power iteration (done, untested) - Add example of tracking throughout training (done, untested) - Add README information on...

This adds a flag to compute the most negative eigenvalues for looking at negative curvature. Resolves #11

Need to ensure that the imports work for each ray trial.

Check / describe support for multi-node experiments with ray. Test these on some basic EC2 instances and make a write-up.