How it works

[abacaj]

I know you linked to the article regarding the algorithm but it would be useful to have a walk-through (high-level) of what the Neuroevolution is doing in the case of this game.

For example, talking about what the inputs are (location on screen?) and what the outputs are (go up, or down).

Thank you, great example.

[sunrei]

As far as I learned from code it takes 2 inputs:

• normalized vertical position of the bird (Y coordinate divided by canvas height)
• normalized vertical position of next gap between pipes (Y coordinate of the bottom edge of top pipe / canvas height)

It doesn't care about bottom pipe and the gap size, and horizontal distance to next gap.

In terms of the game the bird instantly falling down with acceleration (like normal gravity effect), and moving up is supposed to be done by hitting ceretain key on keyboard (e.g. Up or Space) and it looks like jump from the current position of the bird. It's also possible to hold the key pressed and bird will go up with constant speed and upon releasing the key bird performs additional jump.

We have 1 output here - floating number from 0 to 1. If it is greater than 0.5 we assume that bird needs to go up, so it translates it as pressing (or keep holding) UP key, otherwise - releasing the key.

[xviniette]

i will add explanation in README.

[Grabber]

It is a little off-topic, but has anyone tried to apply the underlying concepts to price modeling on e-commerce? I can clearly see an analogy to real transactional application.

[ylin]

@xviniette: I too am trying to understand how the neuroevolution algorithm works. I'm a total beginner to AI, and I wish to understand more about reinforcement learning algorithms like this one.

Is there an academic paper such as the ones on https://arxiv.org that goes over the math behind neuroevolution.js?

Much appreciated.

[NicoJuicy]

Much appreciated if you add explanation in README :)

[kotAPI]

@xviniette Still waiting for the tutorial!

[ylin]

Me 2

On Thu, Aug 17, 2017 at 10:53 PM, Pranay Kothapalli [email protected] wrote: @xviniette [https://github.com/xviniette] Still waiting for the tutorial!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub [https://github.com/xviniette/FlappyLearning/issues/17#issuecomment-323266106] , or mute the thread [https://github.com/notifications/unsubscribe-auth/AATFWwS-cC8UH_VXLgFqW2S1bZ9CSUDpks5sZSbKgaJpZM4Kxfhy] .

[planktonfun]

Cannot quite get XOR to work right bellow 0.3 error rate

[AmrAlaa-exe]

@planktonfun Provide your code, please.

[planktonfun]

@AmrAlaa-exe here you go:

var Neuroevolution = require('./Neuroevolution.js');

//Default options values
var options = {
    network:[2, [2], 1],    // Perceptron structure
    population:50,          // Population by generation
    elitism:0.2,            // Best networks kepts unchanged for the next generation (rate)
    randomBehaviour:0.2,    // New random networks for the next generation (rate)
    mutationRate:0.1,       // Mutation rate on the weights of synapses
    mutationRange:0.5,      // Interval of the mutation changes on the synapse weight
    historic:0,             // Latest generations saved
    lowHistoric:false,      // Only save score (not the network)
    scoreSort:-1,           // Sort order (-1 = desc, 1 = asc)
    nbChild:1               // number of child by breeding
}

// Initialize
var ne = new Neuroevolution(options);

// Generate first or next generation
var generation;

var trainingSet = [
    [[1,1],0],
    [[0,0],0],
    [[0,1],1],
    [[1,0],1]
];

// Error is 100% by default try to minimize it to 0%
var errorpercent = 1;

while(errorpercent >= 0.25) {
    generation = ne.nextGeneration();

    for (var i = options.population - 1; i >= 0; i--) {
        var errorrate = 0;
        for (var j = trainingSet.length - 1; j >= 0; j--) {
            var input = trainingSet[j][0];
            var expected = trainingSet[j][1];

            var result = Math.round(generation[i].compute(input));

            if(result != expected) {
                errorrate += 1;
            }
        }

        errorpercent = errorrate/4;
        ne.networkScore(generation[i], ((errorrate/4)*100)-100);
        console.log(errorpercent*100);
    }
}

console.log(
    Math.round(generation[0].compute([1,1])),
    Math.round(generation[0].compute([0,1])),
    Math.round(generation[0].compute([1,0])),
    Math.round(generation[0].compute([0,0]))
);

[xviniette]

I tried and it works on my end. Try to increase the mutationRange.

[planktonfun]

@xviniette Finally It works when the mutation is at 80% after 3868 iterations thanks! Was starting to lose hope at 400 iterations, and 21282 iterations 50% Mutation.

[ShuhuaGao]

@sunrei Thanks for you explanation. I guess in fact only the relative position matters, i.e., the vertical distance between the bird and the first upper pipe before it. thus, this task can be modeled as a 1-input-1-output function fitting (or binary classification) problem, where the neural networks come in.

[rhysstubbs]

Which Neuroevolution algorithm is this using, is it NEAT? If not, the answer I am really after is whether it evolves the hyper-parameters (i.e. network topology) or not?

[ShuhuaGao]

@rhysstubbs It is not the standard NEAT. Only the network connection weights are evolved, I think. That is, use genetic algorithm instead of back-propagation to train the network.