structural-probes
structural-probes copied to clipboard
Codebase for testing whether hidden states of neural networks encode discrete structures.
Just for the record, in case someone finds it useful or plans to extend it. In line 48 of https://github.com/john-hewitt/structural-probes/blob/4c2e265d6bd071e6ab380fd9806e4c6a128b5e97/scripts/convert_raw_to_bert.py#L48 Segment type ids should be zeros, not ones as implemented...
I am trying to run an experiment on a new dataset. Followed your instructions, but everytime when I run this code, 'AssertionError' appears. I don't know whether it is an...
I noticed that in data.py for BERT you use wordpiece tokenizer instead of full tokenizer. I tried switching to full tokenizer, but strangely it gave me much worse results. I...
Right now, to test a new representation learner, one must: 1. Use the representation learner to write hidden state vectors for each token (or subword) to disk. (Better idea for...