NaNoGenMo
Intent to participate: Teens Wander Around a House, updated
Hi everyone! (Possibly the youngest contributor here.) I plan on reimplementing Teens Wander Around a House in Python, with some GPT-2-generated text layered on top, and the setting changed to the night of Y2k38. That's all.
Update: After manually transpiling @karpathy's code from JS to Python (and re-architecturing some of it along the way), I have a functional code draft here. If you run it, it should eventually print out a TWAAH-ish story on the order of 50k words (possibly fewer or more due to inherent stochasticity). It comes with timestamps, because canonically the narrative is actually written by a "smart home" system observing the teenagers.
Stuff I still have to do to finish:
- Implement the Y2k38 wraparound for the official timestamps while retaining it for the characters.
- Rewrite the helper functions so that they return text instead of printing it.
- Figure out how to call GPT-2 properly (a lot of the GPT-2 libraries on the PyPI don't work for reasons I fail to comprehend).
- Add continuity (on a per-character and per-room basis).
Another update: I've implemented the first two things and placed the output in an actual file. The result is here. Tomorrow night I'll get to working with GPT-2 and continuity.
(Extremely minor note: I've set the mansion in PT because @karpathy was there. Unfortunately, this means that the time jump happens at 7:14pm, which is in the early evening; as the characters arrive in the afternoon and are set to stay there until the next morning, this means that the time jump occurs very early into the novel. I need to figure out how to pick narratively satisfying start and stop times.)
Third update: It looks like GPT-2 is unexpectedly difficult to work with.
Libraries I've tried:
- gpt-2-simple, which initially failed to work. Following this Medium article, however, by downgrading TensorFlow to v1.15, I was able to get the library to load, and I even downloaded the 124M model weights (I got throttled towards the end, but barely made it through). However, generation fails:
(This is me trying to fine-tune GPT-2 on my entry from before. Code taken from documentation.)
-
gpt2-client, which I haven't done much with. However, it too hates the latest versions of TensorFlow, and it also hates when the model comes in size 124M.
-
keras-gpt-2, which requires that one already have model weights to work properly, which is why I initially disliked it. Fortunately, now I have downloaded weights! Unfortunately, kerns-gpt-2 requires TensorFlow version 2.2 or higher.
-
torch-gpt-2, which also requires having the weights. It also errors mysteriously:
I'll let you know if I can get GPT-2 to properly generate text. Feel free to suggest how I could make it work.
Fourth update: Got GPT-2 to work from terminal, but mysteriously not from shell.
After much reflection, I've decided that I want to declare this finished. I already have an output of the required length, and GPT-2 is pretty hard to work with anyways.