Yaae

Yaae copied to clipboard

Introduction

• Yaae (Yet another autodiff engine) is an automatic differentiation engine written in Numpy which comes with a small neural networks library. It supports scalar operations as well as tensors operations and comes with various functions such as exponential, relu, sigmoid ... For educational puprose only.

• Here is my blog post explaining how an automatic differentiation works and my design/implementation choices.

Quickstart

• Let's compare a simple example using Yaae and Pytorch.

# Yaae.
w1 = Node(2, requires_grad=True)
w2 = Node(3, requires_grad=True)
w3 = w2 * w1
w4 = w1.sin()
w5 = w3 + w4
z = w5
z.backward()
w1_yaae, w2_yaae, z_yaae = w1, w2, z

# Pytorch.
w1 = torch.Tensor([2]).double()
w1.requires_grad = True
w2 = torch.Tensor([3]).double()
w2.requires_grad = True
w3 = w2 * w1
w4 = w1.sin()
w5 = w3 + w4
z = w5
z.backward()
w1_torch, w2_torch, z_torch = w1, w2, z

# Forward pass.
assert z_yaae.data == z_torch.data.item() # True.
# Backward pass.
assert w1_yaae.grad.data ==  w1_torch.grad.item() # True.
assert w2_yaae.grad.data == w2_torch.grad.item() # True.

Example of usage

• The files demo_regression.ipynb and demo_classification.ipynb are simple regression/classification problems solved using Yaae. As shown in the notebooks, here are the the results:

• If you are still skeptical, here is my GAN implemented with Yaae.

Installation

• Create a virtual environment in the root folder using virtualenv and activate it.

# On Linux terminal, using virtualenv.
virtualenv my_yaae_env
# Activate it.
source my_yaee_env/bin/activate

• Install requirements.txt.

pip install -r requirements.txt
# Tidy up the root folder.
python3 setup.py clean