htm.core
htm.core copied to clipboard
Python pickle fails for Regions and Network
I would like to multiprocess several parallel networks, in particular the location framework, but this fails on pickle errors for networks and regions. I added pickle tests to the Python bindings unit tests for region and network to check if the problem is me, or is the bindings. It seems it is the bindings unless I am missing something. I have attached the modified unit tests for your convenience.
Good catch.
Serialization of the network is performed by calling network.saveToFile(filename) or save(stream). However, from the Python side a pickle of a network should result in save( ) being called. The stream would need to be captured and copied into the pickle stream. All of this would need to be performed by the bindings. PyBindRegion has code to perform the pickle in the other direction (C++ serializing the python region), but not for a pickle starting on the python side.
For the region, we should have this as well... After a region has been deserialized it would need to be reconnected to the network before it could be used.
@fcr would you like to take on this project?
@dkeeney if I was comfortable enough with C++ I would have been happy to. But it is way too far out of my comfort zone so I must decline.
ok, let me see what I can do.
Thanks. It would be much appreciated.
BTW this is part of my efforts to get around the performance bottleneck of the ColumnPooler.
I am able to reproduce your error:
======================================================================
ERROR: testAL246aCorticalColumnPickle (L46aNetwork_test.TestSimpleSPTMNetwork)
----------------------------------------------------------------------
Traceback (most recent call last):
File "J:\Projects\AI\htmA\py\tests\advanced\regions\L46aNetwork_test.py", line 117, in testAL246aCorticalColumnPickle
network2 = pickle.loads(pickledColumn)
RuntimeError: CHECK FAILED: "srcOutput.get() != o" addLink -- link from region L4 output winnerCells to region L6a input anchorGrowthCandidates already exists
I don't know the cause yet but I will setup another PR to address this.