add AverageLearner1D and AverageLearner2D

[basnijholt]

(original merge request on GitLab)

opened by Bas Nijholt (@basnijholt) at 2018-06-05T21:40:00.078Z

This merge request implements a Learner2D that can learn averages on the points, the AverageLearner2D.

When choosing points the learner can either

• add more values at an existing points
• add more triangles

The learner compares the loss of potential new triangles with the standard error of an existing point.

The relative importance of both can be adjusted by a hyperparameter learner.weight. From the doc-string:

When weight > 1 adding more points to existing points will be prioritized (making the standard error of a point more important,) otherwise adding new triangles will be prioritized (making the loss of a triangle more important.)

All tests that pass for the Learner2D currently pass for the AvererageLearner2D too.

Run with:

import adaptive
adaptive.notebook_extension()

def ring(xys, wait=False):
    import numpy as np
    from time import sleep
    from random import random
    xy, _ = xys
    if wait:
        sleep(random()/100)
    x, y = xy
    a = 0.2
    return (np.arctan((x**2 + y**2 - 0.75**2)/a**2)
            + 10 * np.exp(-(x**2 + y**2 - 0.75**2)**2/a**4) * (random() - 1/2))

learner = adaptive.AverageLearner2D(ring, bounds=[(-1, 1), (-1, 1)], weight=.1)
runner = adaptive.Runner(learner, goal=lambda l: l.loss() < 0.01, log=True)
runner.live_info()

which results in:

>>> print(learner.mean_values_per_point()) 
65.2737642585551328

and

learner.plot(tri_alpha=0.5) + learner.plot_std_or_n(which='std')

• [x] Before merging we should observe that this behaves reasonably when the function is heteroscedastic (noise depends on x).
• [ ] Need to verify that δy between neighbouring points is comparable to std(y). This is best to do in 1D learner.
• [ ] write docstring for AverageLearner1D
• [x] doc-string for AverageLearner2D is from Learner2D for auto complete
• [ ] write doc-strings for properties in reference/adaptive.learner.average1D.html.

[basnijholt]

@anton I have implemented what you suggested in chat:

It's not just that: the two options that make sense are:

• increase the number of samples in a point by a fixed fraction (e.g. 1.1)
• add a new point with the number of samples that's comparable to the number of samples in the neighboring points.

The problem now is that once a point has a lot of "seeds", increasing the number of seeds by 10% will give a big loss improvement, probably the biggest, so the number of values at that point will grow very big. Conceptually this shouldn't happen, so I probably made a mistake in the following method:

    def loss_per_existing_point(self):
        """Increase the number of seeds by 10%."""
        if len(self.data) < 4:
            return [], []
        scale = self.value_scale()
        points = []
        loss_improvements = []

        neighbors = self._get_neighbor_mapping_existing_points()
        mean_values_per_neighbor = self._mean_values_per_neighbor(neighbors)

        for p, sem in self.data_sem.items():
            n_neighbors = mean_values_per_neighbor[p]
            N = self.n_values(p)
            n_more = int(1.1 * N)  # increase the amount of points by 10%
            points.append((p, n_more))
            # This is the improvement considering we will add
            # n_more seeds to the stack.
            sem_improvement = (1 / sqrt(N) - 1 / sqrt(N + n_more)) * sem
            loss_improvement = self.weight * sem_improvement / scale  # XXX: Do I need to divide by the scale?
            loss_improvements.append(loss_improvement)
        return points, loss_improvements

[akhmerov]

The problem now is that once a point has a lot of "seeds", increasing the number of seeds by 10% will give a big loss improvement, probably the biggest, so the number of values at that point will grow very big.

If you increase the number of points by 10%, the rms at the point drops by 5%; why would this be the biggest loss improvement?

[basnijholt]

@akhmerov it is fixed now, there were several bugs.

Everything seems to work now.

I've also renamed the weight to average_priority.

[basnijholt]

I've noticed that the AverageLearner1D/2D aren't working for BalancingLearners because of the ask(..., tell_pending=False) when using bal_learner.strategy = 'loss_improvements'.

[basnijholt]

I've added a cool plotting feature, on hovering over the points it displays extra information:

[basnijholt]

The failing test test_saving[AverageLearner1D-random_linear_with_peak-learner_kwargs6] is because the y_scale not properly being updated.