Chatette
Chatette copied to clipboard
Unnecessary memory error
As far as I can tell, generated examples can be accessed via a generator. However, to write to a file, that generator is turned into a list, which, if that list can cause a memory error.
However, by looking at the source code, it seems that writing to file is done by batch anyway. So why keep all generated examples in memory?
I think this memory error can be easily avoided.
Hi,
This is not an error but rather a preparation: I'd like to make the generation lazy and have the adapters ask for the examples only when they need them. As you point out, this will allow to use less memory. I just didn't have the time to refactor the adapters (and the command line interface) so that they work this way yet.
Feel free to make those changes if you want to.