pymc2
pymc2 copied to clipboard
Published
20 hours ago •
pymc-devs
pymc-devs
HDF5 Output issues with save_interval
I encountered this bug with my own model (https://github.com/choderalab/gbff/issues/10), but the following code reproduces it with the example models in pymc. Removing the save_interval flag fixes the issue.
import pymc
from pymc.examples import disaster_model
sampler = pymc.MCMC(disaster_model, db='hdf5', dbname="./test.h5")
sampler.sample(100, save_interval=1)
Burn-in interval complete
Sampling finished normally.
Warning, unable to save state.
Error message:
Traceback (most recent call last):
File "/home/kyleb/opt/lib/python2.7/site-packages/pymc/Model.py", line 777, in save_state
self.db.savestate(self.get_state())
File "/home/kyleb/opt/lib/python2.7/site-packages/pymc/database/hdf5.py", line 503, in savestate
cur_chain._state_[0] = state
File "/home/kyleb/opt/lib/python2.7/site-packages/tables/vlarray.py", line 785, in __setitem__
self._assign_values(coords, value)
File "/home/kyleb/opt/lib/python2.7/site-packages/tables/vlarray.py", line 718, in _assign_values
nparr[:], exc))
ValueError: Value parameter:
'array([128, 2, 125, 113, 1, 40, 85, 11, 115, 116, 111, 99, 104,
97, 115, 116, 105, 99, 115, 113, 2, 125, 113, 3, 40, 85,
10, 101, 97, 114, 108, 121, 95, 109, 101, 97, 110, 113, 4,
99, 110, 117, 109, 112, 121, 46, 99, 111, 114, 101, 46, 109,
117, 108, 116, 105, 97, 114, 114, 97, 121, 10, 95, 114, 101,
99, 111, 110, 115, 116, 114, 117, 99, 116, 10, 113, 5, 99,
110, 117, 109, 112, 121, 10, 110, 100, 97, 114, 114, 97, 121,
10, 113, 6, 75, 0, 133, 85, 1, 98, 135, 82, 113, 7,
40, 75, 1, 41, 99, 110, 117, 109, 112, 121, 10, 100, 116,
121, 112, 101, 10, 113, 8, 85, 2, 102, 56, 75, 0, 75,
1, 135, 82, 113, 9, 40, 75, 3, 85, 1, 60, 78, 78,
78, 74, 255, 255, 255, 255, 74, 255, 255, 255, 255, 75, 0,
116, 98, 137, 85, 8, 78, 134, 53, 111, 148, 87, 198, 63,
116, 98, 85, 11, 115, 119, 105, 116, 99, 104, 112, 111, 105,
110, 116, 113, 10, 104, 5, 104, 6, 75, 0, 133, 85, 1,
98, 135, 82, 113, 11, 40, 75, 1, 41, 104, 8, 85, 2,
105, 56, 75, 0, 75, 1, 135, 82, 113, 12, 40, 75, 3,
85, 1, 60, 78, 78, 78, 74, 255, 255, 255, 255, 74, 255,
255, 255, 255, 75, 0, 116, 98, 137, 85, 8, 0, 0, 0,
0, 0, 0, 0, 0, 116, 98, 85, 9, 108, 97, 116, 101,
95, 109, 101, 97, 110, 113, 13, 104, 5, 104, 6, 75, 0,
133, 85, 1, 98, 135, 82, 113, 14, 40, 75, 1, 41, 104,
9, 137, 85, 8, 14, 242, 100, 236, 209, 133, 253, 63, 116,
98, 117, 85, 12, 115, 116, 101, 112, 95, 109, 101, 116, 104,
111, 100, 115, 113, 15, 125, 113, 16, 40, 85, 20, 77, 101,
116, 114, 111, 112, 111, 108, 105, 115, 95, 108, 97, 116, 101,
95, 109, 101, 97, 110, 113, 17, 125, 113, 18, 40, 85, 21,
97, 100, 97, 112, 116, 105, 118, 101, 95, 115, 99, 97, 108,
101, 95, 102, 97, 99, 116, 111, 114, 113, 19, 71, 63, 240,
0, 0, 0, 0, 0, 0, 85, 21, 112, 114, 111, 112, 111,
115, 97, 108, 95, 100, 105, 115, 116, 114, 105, 98, 117, 116,
105, 111, 110, 113, 20, 85, 6, 78, 111, 114, 109, 97, 108,
113, 21, 85, 11, 112, 114, 111, 112, 111, 115, 97, 108, 95,
115, 100, 113, 22, 99, 110, 117, 109, 112, 121, 46, 99, 111,
114, 101, 46, 109, 117, 108, 116, 105, 97, 114, 114, 97, 121,
10, 115, 99, 97, 108, 97, 114, 10, 113, 23, 104, 9, 85,
8, 164, 23, 214, 1, 218, 171, 217, 63, 134, 82, 113, 24,
85, 8, 97, 99, 99, 101, 112, 116, 101, 100, 113, 25, 71,
64, 65, 128, 0, 0, 0, 0, 0, 85, 22, 99, 104, 101,
99, 107, 95, 98, 101, 102, 111, 114, 101, 95, 97, 99, 99,
101, 112, 116, 105, 110, 103, 113, 26, 136, 85, 8, 114, 101,
106, 101, 99, 116, 101, 100, 113, 27, 71, 64, 80, 64, 0,
0, 0, 0, 0, 117, 85, 30, 68, 105, 115, 99, 114, 101,
116, 101, 77, 101, 116, 114, 111, 112, 111, 108, 105, 115, 95,
115, 119, 105, 116, 99, 104, 112, 111, 105, 110, 116, 113, 28,
125, 113, 29, 40, 104, 19, 71, 63, 240, 0, 0, 0, 0,
0, 0, 104, 20, 85, 7, 80, 111, 105, 115, 115, 111, 110,
113, 30, 104, 22, 104, 23, 104, 9, 85, 8, 0, 0, 0,
0, 0, 0, 240, 63, 134, 82, 113, 31, 104, 25, 71, 64,
69, 128, 0, 0, 0, 0, 0, 104, 26, 136, 104, 27, 71,
64, 76, 128, 0, 0, 0, 0, 0, 117, 85, 21, 77, 101,
116, 114, 111, 112, 111, 108, 105, 115, 95, 101, 97, 114, 108,
121, 95, 109, 101, 97, 110, 113, 32, 125, 113, 33, 40, 104,
19, 71, 63, 240, 0, 0, 0, 0, 0, 0, 104, 20, 104,
21, 104, 22, 104, 23, 104, 9, 85, 8, 124, 57, 143, 169,
18, 149, 144, 63, 134, 82, 113, 34, 104, 25, 71, 64, 88,
192, 0, 0, 0, 0, 0, 104, 26, 136, 104, 27, 71, 63,
240, 0, 0, 0, 0, 0, 0, 117, 117, 85, 7, 115, 97,
109, 112, 108, 101, 114, 113, 35, 125, 113, 36, 40, 85, 6,
115, 116, 97, 116, 117, 115, 113, 37, 85, 5, 114, 101, 97,
100, 121, 113, 38, 85, 5, 95, 105, 116, 101, 114, 113, 39,
104, 23, 104, 12, 85, 8, 100, 0, 0, 0, 0, 0, 0,
0, 134, 82, 113, 40, 85, 14, 95, 116, 117, 110, 101, 95,
105, 110, 116, 101, 114, 118, 97, 108, 113, 41, 77, 232, 3,
85, 12, 95, 116, 117, 110, 101, 100, 95, 99, 111, 117, 110,
116, 113, 42, 75, 0, 85, 16, 95, 116, 117, 110, 101, 95,
116, 104, 114, 111, 117, 103, 104, 111, 117, 116, 113, 43, 136,
85, 16, 95, 98, 117, 114, 110, 95, 116, 105, 108, 108, 95,
116, 117, 110, 101, 100, 113, 44, 137, 85, 13, 95, 99, 117,
114, 114, 101, 110, 116, 95, 105, 116, 101, 114, 113, 45, 75,
100, 85, 5, 95, 98, 117, 114, 110, 113, 46, 75, 0, 85,
5, 95, 116, 104, 105, 110, 113, 47, 75, 1, 117, 117, 46], dtype=uint8)'
cannot be converted into an array object compliant vlarray[0] row:
'array([128, 2, 125, 113, 1, 40, 85, 11, 115, 116, 111, 99, 104,
97, 115, 116, 105, 99, 115, 113, 2, 125, 113, 3, 40, 85,
10, 101, 97, 114, 108, 121, 95, 109, 101, 97, 110, 113, 4,
99, 110, 117, 109, 112, 121, 46, 99, 111, 114, 101, 46, 109,
117, 108, 116, 105, 97, 114, 114, 97, 121, 10, 95, 114, 101,
99, 111, 110, 115, 116, 114, 117, 99, 116, 10, 113, 5, 99,
110, 117, 109, 112, 121, 10, 110, 100, 97, 114, 114, 97, 121,
10, 113, 6, 75, 0, 133, 85, 1, 98, 135, 82, 113, 7,
40, 75, 1, 41, 99, 110, 117, 109, 112, 121, 10, 100, 116,
121, 112, 101, 10, 113, 8, 85, 2, 102, 56, 75, 0, 75,
1, 135, 82, 113, 9, 40, 75, 3, 85, 1, 60, 78, 78,
78, 74, 255, 255, 255, 255, 74, 255, 255, 255, 255, 75, 0,
116, 98, 137, 85, 8, 78, 134, 53, 111, 148, 87, 198, 63,
116, 98, 85, 11, 115, 119, 105, 116, 99, 104, 112, 111, 105,
110, 116, 113, 10, 104, 5, 104, 6, 75, 0, 133, 85, 1,
98, 135, 82, 113, 11, 40, 75, 1, 41, 104, 8, 85, 2,
105, 56, 75, 0, 75, 1, 135, 82, 113, 12, 40, 75, 3,
85, 1, 60, 78, 78, 78, 74, 255, 255, 255, 255, 74, 255,
255, 255, 255, 75, 0, 116, 98, 137, 85, 8, 0, 0, 0,
0, 0, 0, 0, 0, 116, 98, 85, 9, 108, 97, 116, 101,
95, 109, 101, 97, 110, 113, 13, 104, 5, 104, 6, 75, 0,
133, 85, 1, 98, 135, 82, 113, 14, 40, 75, 1, 41, 104,
9, 137, 85, 8, 14, 242, 100, 236, 209, 133, 253, 63, 116,
98, 117, 85, 12, 115, 116, 101, 112, 95, 109, 101, 116, 104,
111, 100, 115, 113, 15, 125, 113, 16, 40, 85, 20, 77, 101,
116, 114, 111, 112, 111, 108, 105, 115, 95, 108, 97, 116, 101,
95, 109, 101, 97, 110, 113, 17, 125, 113, 18, 40, 85, 21,
97, 100, 97, 112, 116, 105, 118, 101, 95, 115, 99, 97, 108,
101, 95, 102, 97, 99, 116, 111, 114, 113, 19, 71, 63, 240,
0, 0, 0, 0, 0, 0, 85, 21, 112, 114, 111, 112, 111,
115, 97, 108, 95, 100, 105, 115, 116, 114, 105, 98, 117, 116,
105, 111, 110, 113, 20, 85, 6, 78, 111, 114, 109, 97, 108,
113, 21, 85, 11, 112, 114, 111, 112, 111, 115, 97, 108, 95,
115, 100, 113, 22, 99, 110, 117, 109, 112, 121, 46, 99, 111,
114, 101, 46, 109, 117, 108, 116, 105, 97, 114, 114, 97, 121,
10, 115, 99, 97, 108, 97, 114, 10, 113, 23, 104, 9, 85,
8, 164, 23, 214, 1, 218, 171, 217, 63, 134, 82, 113, 24,
85, 8, 97, 99, 99, 101, 112, 116, 101, 100, 113, 25, 71,
64, 65, 128, 0, 0, 0, 0, 0, 85, 22, 99, 104, 101,
99, 107, 95, 98, 101, 102, 111, 114, 101, 95, 97, 99, 99,
101, 112, 116, 105, 110, 103, 113, 26, 136, 85, 8, 114, 101,
106, 101, 99, 116, 101, 100, 113, 27, 71, 64, 80, 64, 0,
0, 0, 0, 0, 117, 85, 30, 68, 105, 115, 99, 114, 101,
116, 101, 77, 101, 116, 114, 111, 112, 111, 108, 105, 115, 95,
115, 119, 105, 116, 99, 104, 112, 111, 105, 110, 116, 113, 28,
125, 113, 29, 40, 104, 19, 71, 63, 240, 0, 0, 0, 0,
0, 0, 104, 20, 85, 7, 80, 111, 105, 115, 115, 111, 110,
113, 30, 104, 22, 104, 23, 104, 9, 85, 8, 0, 0, 0,
0, 0, 0, 240, 63, 134, 82, 113, 31, 104, 25, 71, 64,
69, 128, 0, 0, 0, 0, 0, 104, 26, 136, 104, 27, 71,
64, 76, 128, 0, 0, 0, 0, 0, 117, 85, 21, 77, 101,
116, 114, 111, 112, 111, 108, 105, 115, 95, 101, 97, 114, 108,
121, 95, 109, 101, 97, 110, 113, 32, 125, 113, 33, 40, 104,
19, 71, 63, 240, 0, 0, 0, 0, 0, 0, 104, 20, 104,
21, 104, 22, 104, 23, 104, 9, 85, 8, 124, 57, 143, 169,
18, 149, 144, 63, 134, 82, 113, 34, 104, 25, 71, 64, 88,
192, 0, 0, 0, 0, 0, 104, 26, 136, 104, 27, 71, 63,
240, 0, 0, 0, 0, 0, 0, 117, 117, 85, 7, 115, 97,
109, 112, 108, 101, 114, 113, 35, 125, 113, 36, 40, 85, 6,
115, 116, 97, 116, 117, 115, 113, 37, 85, 7, 114, 117, 110,
110, 105, 110, 103, 113, 38, 85, 5, 95, 105, 116, 101, 114,
113, 39, 104, 23, 104, 12, 85, 8, 100, 0, 0, 0, 0,
0, 0, 0, 134, 82, 113, 40, 85, 14, 95, 116, 117, 110,
101, 95, 105, 110, 116, 101, 114, 118, 97, 108, 113, 41, 77,
232, 3, 85, 12, 95, 116, 117, 110, 101, 100, 95, 99, 111,
117, 110, 116, 113, 42, 75, 0, 85, 16, 95, 116, 117, 110,
101, 95, 116, 104, 114, 111, 117, 103, 104, 111, 117, 116, 113,
43, 136, 85, 16, 95, 98, 117, 114, 110, 95, 116, 105, 108,
108, 95, 116, 117, 110, 101, 100, 113, 44, 137, 85, 13, 95,
99, 117, 114, 114, 101, 110, 116, 95, 105, 116, 101, 114, 113,
45, 75, 99, 85, 5, 95, 98, 117, 114, 110, 113, 46, 75,
0, 85, 5, 95, 116, 104, 105, 110, 113, 47, 75, 1, 117,
117, 46], dtype=uint8)'
The error was: <could not broadcast input array from shape (936) into shape (938)>
System details: anaconda python2.7, pytables 3.1.1, Ubuntu 14.04. I've tried both pymc 2.3.4 and latest git.