Memory usage and optimization

[c4b4d4]

Hello @DavidCai1993, I cloned your repo to learn about training an algorithm to generate sentences. And I'm glad I found it, thanks for sharing it.

I'm having a little issue here. I've seen that you use const batchSize = 3000 to train the algorithm, but this increases a lot the memory usage, at least on my configured setup. I own a 2070 RTX.

How do you manage to train a batch so big?

The maximum I manage to train is a batch size of 350. Bigger than this number I always end up consuming the whole 8GB of my GPU.