neat-python
neat-python copied to clipboard
CodeReclaimers
set bounds and constraints on each of the output nodes
So let's say that I want the output nodes 0 to 3 are some kind of probabilities that will be bounded between 0 and 1 (not simply clamped) and I want their sum to add up to 1.0, and then I want node 4 to be a value between 55 and 140.
I believe some solutions might be found a significant amount faster if the problems are set up with these types of configurations similar to how scipy.optimize.minimize works:
bounds = [(0,1), (0,1), (0,1), (0,1), (55,140)] constraints = [{ 'type': 'eq', 'fun': lambda nodes: 1.0 - np.sum(nodes) }] with one more parameter that will be a list of which nodes to use [0,1,2,3] in this example
These could be part of the configuration.
I usually just code stuff like scaling in the genome evaluation function, but it would be nice to have some standard machinery available to avoid having to roll it from scratch every time.
What I usually have seen done in the past for the "sum to 1" constraint is to use softmax (https://en.wikipedia.org/wiki/Softmax_function). One of the nice things (I suppose) about using something like that is you just have to write the forward function without worrying about computing derivatives for backprop. :)
@stark7 Unless you hit some precision issues, why not just normalizing all input/output data so it is between 0 and 1 ?
the bounds in the example above were 0,1 and then 55,140 . Also modifying the output coming out of the NN might defeat the learning process - I am hoping for a way to keep it all encapsulated in the NN.
May be I am still missing something but then inputs would look like that: {0/55, 1/140} another words input_A = a/max(a_inputs) input_B = a/max(b_inputs), this way nothing modified just normalized.
I'm asking about bounds and constraints on the output nodes only, not the input. The input nodes should be whatever comes in, but the output nodes is what should be bounded and/or constrained.
Normalizing/Denormalizing Example: Normalizing input: node_input = (input - min(inputs)) / (max(inputs) - min(inputs)) Denormalizing output: node_output = min_bound + node_output*(max_bound - min_bound)
Above example is most basic one, there are many ways to normalize/standardize data that would be project specific, to minimize information loss or optimize data processing.
Thanks for the quick follow-ups d0pa - I think I follow what you're saying. Do you think it's a good idea to build in the neat-python lib this mechanism you wrote? : node_output = min_bound + node_output*(max_bound - min_bound) . I'm actually still quite new at this and trying to learn the best ways to setup a problem to be solved.
I think its kind of a separate project in by itself pre/post processing can be pretty complex and in my experience always specific to a problem. Will add normalizing example in next neat mpi update. I would suggest do not think in terms of best way, simply just hack project together and solve your main bottlenecks and/or issues.
I'll follow that suggestion. Lately I've been having the same issue over and over and that has to do with bounds and constraints on the output. My hope was that adding bounds and constraints would also reduce the learning time complexity of neat.
EDIT: I just thought of something - what if we use some kind of built-in fitness calculation based on these bounds and constraints that will basically help reject all genomes that don't fall between the bounds?
After some testing and trying things out, I can't have a normalization steps before and after because of computational complexity my deployment device where the NN does something. The entire processing should be done within the activate def using the computed weights and biases. So the bounds and constraints should be used during the learning process.
I am working on adding these myself only I don't know enough about the neat-python guts and glory yet it seems to make this happen.
As @d0pa says, my experience is usually that scaling and normalization of parameters are usually application-specific, so I haven't spent a lot of time coming up with something to handle a case like this. NEAT can conceivably evolve the appropriate transforms for any input or output if you let it use unbounded activation functions like relu and identity, but as you point out it sometimes takes it a long time to figure those out, and it's better to manually implement that stuff yourself if you know it in advance.
One thing you could do is to add a custom sigmoid- or tanh-like activation function that has a range that you need on the outputs. For example, if your application only accepts outputs on [0, 5], you could register a new activation function that looks like this:
def zero_to_five_activation(z):
return 2.5 * (1 + math.tanh(z))
So, is there a way to generate these custom bounded functions automatically based on an input and output configurations as described in my first post above?
I am wondering if the constraints can be thought of as a custom aggregation functions and then we basically get exactly what I was asking about.
I think this can be a feasible and useful feature so we don't have to think too hard about configuring things.
I think we could put some helper functions somewhere that would auto-create some input and output transforms based on user-provided ranges. If I can find some time I'll try to get that in there. :)