HierarchicalTemporalMemory.jl icon indicating copy to clipboard operation
HierarchicalTemporalMemory.jl copied to clipboard

Rethink SP local inhibition

Open Oblynx opened this issue 4 years ago • 1 comments

The SP tiebreaker is necessary to enforce strict sparsity control, but it has undesirable sideffects in other properties:

  • sp(z) becomes stochastic (random seed from hash(z)?)
  • noise robustness suffers and doesn't improve a lot with training

This points to a need to rethink inhibition, and not do it by comparing against a threshold, but more low-level, by ordering indices and picking the highest activations.

Oblynx avatar Jul 29 '20 12:07 Oblynx

Implemented in this commit, but only for global inhibition.

Oblynx avatar Sep 05 '21 23:09 Oblynx