Kevin Clark

Hi! Using a smaller generator should work better; we used a larger generator for ELECTRA-Small++ (the released ELECTRA-Small model) on accident. This may have hurt its performance a bit, but...

That's right. See Figure 3 in our paper for some results with different generator sizes.

As the README says, download the Yelp academic dataset (you can get it [here](https://www.yelp.com/dataset_challenge)), put it in the data/provided directory, and run dataset_maker.py. This will create the review.json file.

The feature extraction is implemented in java as part of [CoreNLP](http://stanfordnlp.github.io/CoreNLP/). You can see the feature extraction code [here](https://github.com/stanfordnlp/CoreNLP/blob/master/src/edu/stanford/nlp/coref/neural/CategoricalFeatureExtractor.java).

Huh, that strange. I trained the model on a 128G machine, but I don't think it should use more than 10G RAM. Do you know what's taking up all the...