No validation/test?

[totuta]

Hi,

Looks like your code doesn't have validation or testing part. Hope it can use some fraction of input.txt for valid/test purpose. Any plan?

[fujimotomh]

To anyone interested, this fork has validation and test splitting implemented like Justin Johnson's torch-rnn as well as some other things like dropout, state as tuple, and sampling with temperature. I redid many names and the arrangement in the fork, but I hope it's easy to decipher. Also I'd be happy to make a pull request if there's interest.

[fujimotomh]

Hi Alexander. I changed some of the layout of the code from the original author. I separated the preprocess step from training. The files are created in preprocess.py.

On Oct 27, 2016 10:53 AM, "Alexander" [email protected] wrote:

@fujimotomh https://github.com/fujimotomh I might have missed it, but in your branch, where is the code that creates all those files?

    vocab_file = os.path.join(self.args.data_dir, "vocab.pkl")
    train_file = os.path.join(args.data_dir, "train.npy")
    val_file = os.path.join(args.data_dir, "val.npy")
    test_file = os.path.join(args.data_dir, "test.npy")

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/sherjilozair/char-rnn-tensorflow/issues/33#issuecomment-256720528, or mute the thread https://github.com/notifications/unsubscribe-auth/AUXJkx9VDpR3noty2-nmaS39WX1JD4Kbks5q4OUcgaJpZM4I1yOb .

[kaihuchen]

@fujimotomh
Thanks for making your fork available. I find what you did with separating out some functions (such as pre-processing) quite useful, since it makes it easier for me to experiment with refactoring the raw input with higher-level "characters" (representing substrings) in multi-passes for discovering more abstract patterns.

[ubergarm]

@fujimotomh I like how you've cleaned up a lot of the code in your fork.

It may be possible to pull in some of your work back to this repo. The easiest might be to just get the temperature stuff from sample.py back. The other features might take more work now that the forks have diverged.

Anyway, thanks for sharing!