keras-molecules
keras-molecules copied to clipboard
maxhodak
Add random seed of training for reproducibility
Could you also change the generator-based approach? Specifically, there is a random.shuffle call here that can be seeded: https://github.com/maxhodak/keras-molecules/blob/master/molecules/vectorizer.py. Additionally, perhaps seed should be a flag that can be passed into train.py and train_gen.py.
I don't think it's a good thing to do this (I mean add flag) , according to the comment here, by default Keras's model.compile() sets the shuffle argument as True. You should the set numpy seed before importing keras. Then, adding a flag would make code messy.
Oh, I got it. However, I think it's two different things. Random seed for numpy and random seed for random.
I meant something like that SmilesDataGenerator.init could take a seed kwarg, which would be passed in at train_gen.py, using some sort of flag. In that case, it's ok to pass it in as acquired from a flag. Regarding train.py, you could move the keras import statements into the main function, with argparsing of whether there is a seed flag or not.
Let's also pull the requirements commit out of this PR, and I'll accept it in the other PR.
I've committed a couple changes to train and train_gen to take the seed as a cli parameter. Could you test that they work as you expect?
I test using tensorflow backend, and I found it doesn't work. After a quick search of this, it seems still open issue. Besides, I failed to execute using theano backend. I am still checking what happened.
Do you at least get consistent shuffling? Is it just the weight initialization that is not seeded?
On Fri, Dec 23, 2016 at 9:21 AM, hsiao yi [email protected] wrote:
I test using tensorflow backend, and I found it doesn't work. After a quick search of this, it seems still open issue https://github.com/fchollet/keras/issues/2280. Besides, I failed to execute using theano backend. I am still checking what happened.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/maxhodak/keras-molecules/pull/51#issuecomment-268996757, or mute the thread https://github.com/notifications/unsubscribe-auth/AFGDhjBjitQKZOHcEO4NUhnAhsVSS7-pks5rK9jSgaJpZM4LSxtK .
It looks like the train.py works fine using theano backend, so I think we can wait the keras update in the future. I will now start checking the train_gen.py part.
I am still unable to execute the train_gen.py. What is the input of this file?
Oh, I find that I used wrong input file. The train_gen.py doesn't need preprocessing.py.
#2743 is still an issue for me, despite all the suggestions. Restarting the notebook gives different results as well.
keras 2.0.2 numpy 1.11.2 tensorflow 1.0.0
It turns out that it is due to fchollet/keras#2280. I gave up trying these things long time ago. It seems that it can't be fixed in an easy manner.
Thanks for the information. Non-reproducible results is a serious issue in keras imo.