neat-python icon indicating copy to clipboard operation
neat-python copied to clipboard
verified publisher icon CodeReclaimers

All TODO @ one place

Open mdalvi opened this issue 8 years ago • 0 comments

This is not an issue.

This is to help contributors @CodeReclaimers, @drallensmith, @bennr01, @evolvingmeat, a glimpse at all the TODO's found in this repository.

Code related

@ Module summaries

aggregations - AggregationFunctionSet

  • [ ] TODO: Check for whether the function needs reduce <functools.reduce>, or at least offer a form of this function (or extra argument for it, defaulting to false) and/or its interface in :py:mod:genome, that will appropriately "wrap" the input function.

aggregations - validate_aggregation

  • [ ] TODO: Recognize when need reduce

attributes

  • [ ] TODO: There is probably a lot of room for simplification of these classes using metaprogramming.

attributes - StringAttribute - mutate_value

  • [ ] TODO: Add configurable probabilities of which option is used. Longer-term, as with the improved version of RBF-NEAT, separate genes for the likelihoods of each (but always doing some change, to prevent overly-conservative evolution due to its inherent short-sightedness), allowing the genomes to control the distribution of options, will be desirable.

config - UnknownConfigItemError

  • [ ] TODO: :py:class:genome.DefaultGenomeConfig does not currently check for these.

distributed - _ExtendedManager - reduce

  • [ ] TODO: Appears to assume that start (for initialization) should be true; perhaps self.manager should be checked? (This may require :py:meth::stop() to set self.manager to None, incidentally.)

distributed - _ExtendedManager - stop

  • [ ] TODO: Should this set self.manager to None?

distributed - DistributedEvaluator - evaluate

  • [ ] TODO: Improved handling of errors from broken connections with the secondary nodes may be needed.

neat\genes.py

  • [ ] TODO: There is probably a lot of room for simplification of these classes using metaprogramming.

neat\genes.py

  • [ ] TODO: Evaluate using slots for performance/memory usage improvement.

neat\genes.py

  • [ ] TODO: Should these be in the nn module? iznn and ctrnn can have additional attributes.

neat\genes.py

  • [ ] TODO: Do an ablation study to determine whether the enabled setting is important--presumably mutations that set the weight to near zero could provide a similar effect depending on the weight range, mutation rate, and aggregation function. (Most obviously, a near-zero weight for the product aggregation function is rather more important than one giving an output of 1 from the connection, for instance!)

genome - DefaultGenomeConfig

  • [ ] TODO: Check for unused configuration parameters from the config file.

genome - DefaultGenomeConfig - get_new_node_key

  • [ ] TODO: Explore using the same :term:node key if a particular connection is replaced in more than one genome in the same generation (use a :py:meth:reporting.BaseReporter.end_generation method to wipe a dictionary of connection tuples versus node keys).

genome - DefaultGenomeConfig - add_connection

  • [ ] TODO: Add further validation of this connection addition?

genome - DefaultGenomeConfig - mutate_add_connection

  • [ ] TODO: Should this be using mutation to/from rates? Hairy to configure...

genome - DefaultGenomeConfig - mutate_delete_connection

  • [ ] TODO: If the connection is :term:enabled, have an option to - possibly with a :term:weight-dependent chance - turn its :term:enabled attribute to False instead.

neat\ctrnn_init_.py - get_max_time_step

  • [ ] TODO: Compute max time step that is known to be numerically stable forthe current network configuration. pylint: disable=no-self-use

neat\iznn_init_.py

  • [ ] TODO: Add mechanisms analogous to axon & dendrite propagation delay.

neat\iznn_init_.py - IZNeuron - advance

  • [ ] TODO: Make the time step adjustable, and choose an appropriate numerical integration method to maintain stability.

neat\iznn_init_.py - IZNeuron - advance

  • [ ] TODO: The need to catch overflows indicates that the current method is not stable for all possible network configurations and states.

neat\iznn_init_.py - IZNodeGene

  • [ ] TODO: Genomic distance currently does not take into account the node's :term:bias.

neat\iznn_init_.py - IZNeuron

  • [ ] TODO: Currently has some numerical stability problems.

neat\iznn_init_.py - IZNN

  • [ ] TODO: Investigate this (particularly effects on numerical stability issues).

neat\iznn_init_.py - IZNN - get_time_step_msec

  • [ ] TODO: Investigate performance or numerical stability issues that may result from using this hard-coded time step.

reporting

  • [ ] TODO: Add a curses-based reporter.

reproduction - DefaultReproduction

  • [ ] TODO: Provide some sort of optional cross-species performance criteria, which are then used to control stagnation and possibly the mutation rate configuration. This scheme should be adaptive so that species do not evolve to become "cautious" and only make very slow progress.

reproduction - DefaultReproduction - reproduce

  • [ ] TODO: Make minimum difference configurable (defaulting to 1 to preserve compatibility).

reproduction - DefaultReproduction - reproduce

  • [ ] TODO: I don't like this modification of the species and stagnation objects, because it requires internal knowledge of the objects.

reproduction - DefaultReproduction - reproduce

  • [ ] TODO: The 1.0 below is rather arbitrary, and should be configurable.

reproduction - DefaultReproduction - reproduce

  • [ ] TODO: Isn't the effective min_species_size going to be max(min_species_size, self.reproduction_config.elitism)? That would probably produce more accurate tracking of population sizes and relative fitnesses...

neat\six_util.py

  • [ ] TODO: Perhaps rename this to platform.py or something and add OS-specific hardware detection.

species - DefaultSpeciesSet - speciate

  • [ ] TODO: The current code has a docstringstating that there may be a problem if all old species representatives are not dropped for each generation; it is not clear how this is consistent with the code in :py:meth:reproduction.DefaultReproduction.reproduce, such as for :ref:elitism <elitism-label>.

species - DefaultSpeciesSet - speciate

  • [ ] TODO: Check if sorting the unspeciated genomes by fitness will improve speciation (by making the highest-fitness member of a species its representative).

stagnation

  • [ ] TODO: Add a method for the user to change the "is stagnant" computation.

stagnation

  • [ ] TODO: Currently, depending on the settings for :ref:species_fitness_func <species-fitness-func-label> and :ref:fitness_criterion <fitness-criterion-label>, it is possible for a species with members above the :ref:fitness_threshold <fitness-threshold-label> level of fitness to be considered "stagnant" (including, most problematically, because they are at the limit of fitness improvement).

stagnation - DefaultStagnation - update

  • [ ] TODO: Currently interacts directly with the internals of the :py:class:species.Species object. Also, currently both checks for num_non_stagnant to stop marking stagnant and does not allow the top species_elitism species to be marked stagnant. While the latter could admittedly help with the problem mentioned above, the ordering of species fitness is using the fitness gotten from the species_fitness_func (and thus may miss high-fitness members of overall low-fitness species, depending on the function in use).

statistics

  • [ ] TODO: Currently keeps accumulating information in memory, which may be a problem in long runs.

threaded

  • [ ] TODO: Ensure that there are no reference-cycles. Avoid reference cycles to ensure this method is called. (Perhaps use weakref, depending on what the cycles are? Note that weakref is not compatible with saving via pickle, so all of them will need to be removed prior to any save.)

@ Examples

examples\circuits\evolve.py - mutate

  • [ ] TODO: Make a configuration item to choose whether or not multiple mutations can happen simultaneously.

examples\circuits\evolve.py - add_connection

  • [ ] TODO: Add validation of this connection addition.

examples\picture2d\evolve_interactive.py - make_thumbnails

  • [ ] TODO: This code currently generates the image data using the multiprocessing pool, and then does the actual image construction here because pygame complained about not being initialized if the pool workers tried to construct an image.# Presumably there is some way to fix this, but for now this seems fast enough for the purposes of a demo.

@ Tests

tests\test_activation.py

  • [ ] TODO: These tests are just smoke tests to make sure nothing has become badly broken. Expand to include more detailed tests of actual functionality.

tests\test_aggregation.py

  • [ ] TODO: These tests are just smoke tests to make sure nothing has become badly broken. Expand to include more detailed tests of actual functionality.

tests\test_config.py - test_bad_config_activation

  • [ ] TODO: fix this test

tests\test_feedforward_network.py - test_simple_nohidden

  • [ ] TODO: Update this test for the current implementation.

tests\test_feedforward_network.py - test_simple_hidden

  • [ ] TODO: Update this test for the current implementation.

tests\test_genome.py - test_fs_neat_hidden_old

  • [ ] TODO: Test for emitted warning

tests\test_genome.py - test_fully_connected_hidden_nodirect_old

  • [ ] TODO: Test for emitted warning

tests\test_genome.py - test_partially_connected_hidden_nodirect_old

  • [ ] TODO: Test for emitted warning

tests\test_iznn.py - test_iznn_evolve

  • [ ] TODO: Update this test to work with the current implementation.

tests\test_iznn.py - test_iznn_evolve

  • [ ] TODO: This is a little hackish, but will a user ever want to do it? If so, provide a convenience method on Config for it.

tests\test_simple_run.py - test_run_iznn

  • [ ] TODO: Takes the longest of any of the tests in this file, by far. Why? Was because had population of 290 thanks to too much speciation - too-high compatibility_weight_coefficient relative to range for weights.

tests\test_utils.py

  • [ ] TODO: These tests are just smoke tests to make sure nothing has become badly broken. Expand to include more detailed tests of actual functionality.

Documentation related

@ Customization Behaviour

Spciation scheme

  • [ ] TODO: Further document species set interface (some done in module_summaries)
  • [ ] TODO: Include example

Reproduction scheme

  • [ ] TODO: Include example

@ Glossary

enabled

  • [ ] TODO: Some versions of NEAT give a chance, such as 25%, that a disabled connection will be enabled during :term:crossover; in the future, this should be an option.

@ Index

NEAT-Python's documentation

  • [ ] TODO: Improve README.md file information for the examples.

mdalvi avatar Nov 13 '17 12:11 mdalvi