Using hyperas hangs pool.map()

[chrwang]

I'm using a custom data loading function to read in data to a neural net for hyperas. Since there are many large files, the function creates multiple threads to do so. The function runs fine in a normal python script, but in a hyperas optim session the pool.map() call hangs.

def data():
    from keras.utils import np_utils
    from sklearn.preprocessing import LabelEncoder
    from sklearn.model_selection import train_test_split
    from sasnet import read_seq_1d
    a, b, c, d = read_parallel_1d("../sasmodels/out2/", pattern="_eval_", verbosity=True)
    encoder = LabelEncoder()
    encoder.fit(c)
    encoded = encoder.transform(c)
    yt = np_utils.to_categorical(encoded)
    xtrain, xtest, ytrain, ytest = train_test_split(b, yt, test_size=.25,random_state=235)
    return xtrain, ytrain, xtest, ytest

The gist of read_parallel_1d is here The full program is here

This is running on Python 2.7.2, Keras 2.0.5, tensorflow-gpu 1.2, and hyperas 0.4.

Any thoughts?