pytorch-hebbian

pytorch-hebbian copied to clipboard

PyTorch Hebbian

A lightweight framework for Hebbian learning based on PyTorch Ignite. Presented at the Beyond Backpropagation NeurIPS 2020 worskhop.

Table of Contents

• About the Project
• Getting Started
• Usage
• Contributing
• License
• Reference
• Contact
• Acknowledgements

About the project

A lightweight and flexible framework for Hebbian learning in PyTorch.

Getting started

To get a local copy up and running follow these simple steps.

Prerequisites

Depending on your system (OS/GPU/CUDA support) you may need to manually install a specific PyTorch version. Please see the PyTorch website for more information.

Installation

Clone the repository:

pip install git+https://github.com/Joxis/pytorch-hebbian.git

Usage

Similar to a classic PyTorch pipeline, a model, dataset and data loader are first initiated. Next, the framework requires you to specify a learning rule, optimizer and trainer. You can then start Hebbian learning.

Simple example

# Creating the model
model = models.create_fc1_model([28 ** 2, 400])

# Creating the dataset and data loaders
dataset = datasets.mnist.MNIST(root=config.DATASETS_DIR, download=True,
                               transform=transforms.ToTensor())
train_loader = DataLoader(dataset, batch_size=1024, shuffle=True)

# Creating the learning rule, optimizer and trainer
learning_rule = KrotovsRule()
optimizer = Local(named_params=model.named_parameters(), lr=0.01)
trainer = HebbianTrainer(model=model, learning_rule=learning_rule,
                         optimizer=optimizer)

# Running the trainer
trainer.run(train_loader=train_loader, epochs=10)

Visualizations

Various TensorBoard visualizations are supported. Below are some examples.

See examples for more.

Contributing

Any contributions you make are greatly appreciated. For major changes, please open an issue first to discuss what you would like to change.

1. Fork the project
2. Create your feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add some amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a pull request

License

Distributed under the MIT License. See LICENSE for more information.

Reference

Please consider citing this work if it helps your research.

@inproceedings{talloen2020pytorchhebbian,
  author       = {Jules Talloen and Joni Dambre and Alexander Vandesompele},
  location     = {Online},
  title        = {PyTorch-Hebbian: facilitating local learning in a deep learning framework},
  year         = {2020},
}

Contact

Jules Talloen - <firstname>@<lastname>.eu

Project Link: https://github.com/Joxis/pytorch-hebbian

Acknowledgements

• Ghent University - IDLab
• ML6