Full crash on regen while viewing cells

[celiasmith]

Nengo completely stops if you do the following:

1. add a network, a population with 2 neurons, a termination, and a function input (connect stuff)
2. run the network and show graphs (e.g. slider, value for input and value, spike raster (all neurons), and tuning curves for population)
3. leave the network running, and go back to the creation window.
4. click on the population, bring up the inspector, and change the number of neurons to something else (e.g. 4)

For me this complete freezes the program and nothing works. I have tried to reproduce this error by adding one thing at a time while running, and haven't been able to get a simpler crash case (yet).

You get an error like:

Exception in thread "Thread-48" at ca.nengo.model.impl.EnsembleOrigin.composeSpikeOutput(EnsembleOrigin.java:141) at ca.nengo.model.impl.EnsembleOrigin.getValues(EnsembleOrigin.java:105) at sun.reflect.GeneratedMethodAccessor90.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597)

OR... Exception in thread "Thread-36" Traceback (most recent call last): File "python/timeview/view.py", line 761, in run self.timelog.tick() File "python/timeview/view.py", line 761, in run self.timelog.tick() File "python/timeview/timelog.py", line 113, in tick for item in self.items: item.tick(limit=self.tick_limit) File "python/timeview/timelog.py", line 27, in tick if len(v)<self.length: v=v+[0]*(self.length-len(v)) TypeError: unsupported operand type(s) for +: 'array.array' and 'list'

[studywolf]

I'm unable to reproduce this on Ubuntu...

[studywolf]

That said, when I change the number of neurons nothing is changed, not even the number of neurons in the title above the population. Perhaps we should make this not possible.

[celiasmith]

Unfortunately it is often convenient to change the number of neurons... but yeah, if it's causing lots of problems we should probably just force people to remake the population...

[studywolf]

Yay for lots of posts! Apologies for cluttering.

If I display the tuning curves or population value, I don't get an error, but with the spike raster, voltage graph, firing rates, anything plotting individual neuron activity I get this:

Exception in thread "Thread-12" at ca.nengo.model.impl.EnsembleOrigin.composeSpikeOutput(EnsembleOrigin.java:148) at ca.nengo.model.impl.EnsembleOrigin.getValues(EnsembleOrigin.java:112) at sun.reflect.GeneratedMethodAccessor127.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601)

ca.nengo.model.SimulationException: ca.nengo.model.SimulationException: Some of the Node Origins are not producing spiking output

Even without a termination or function input.

[tcstewar]

Yikes -- I'm surprised that works at all.

For me (on Windows 7), I can recreate this crash, but I have to try it a few times. Just leave the interactive plots running and keep changing the number of neurons in the population. It took me a few tries, but it'll crash. Try going to a larger number of neurons (so that it takes longer to do the re-creation.

My guess is that what's happening is that if step() is called while the NEFEnsembleImpl is still in the process of regenerating itself, then this can happen. Basically, setNodeCount isn't threadsafe.

Any thoughts on this? It's been a long time since I've played with Java's "synchronized" stuff....

Here's my error message -- note that the error comes from Thread-6 while MainThread is in the process of building the gamma matrix....

INFO  [MainThread:ca.nengo.math.impl.WeightedCostApproximator]: Using 10 singula
r values for pseudo-inverse
INFO  [MainThread:ca.nengo.util.Memory]: Used: 55545392 Total: 101515264 Max: 81
0942464 (after inverse)
Exception in thread "Thread-6"  at ca.nengo.model.impl.EnsembleOrigin.composeSpi
keOutput(EnsembleOrigin.java:148)
        at ca.nengo.model.impl.EnsembleOrigin.getValues(EnsembleOrigin.java:112)

        at sun.reflect.GeneratedMethodAccessor115.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)

ca.nengo.model.SimulationException: ca.nengo.model.SimulationException: Some of
the Node Origins are not producing spiking output
INFO  [MainThread:ca.nengo.util.Memory]: Used: 54353256 Total: 101515264 Max: 81
0942464 (before gamma)
INFO  [MainThread:ca.nengo.util.Memory]: Used: 54355392 Total: 101515264 Max: 81
0942464 (before inverse)

[e2crawfo]

you're right Terry, on Ubuntu if I just throw "synchronized" onto the setNodeCount method, the error seems to vanish, unless I go down to 1 neuron. In that case, it always gives the second of the two error messages that Chris reported.

I think technically some other methods should also be synchronized, not sure which ones yet. Also with this, most of the interactive plots (e.g. tuning curves) aren't updated properly after the number of nodes is changed.