genienlp icon indicating copy to clipboard operation
genienlp copied to clipboard

Support for non-almond datasets

Open Mehrad0711 opened this issue 2 years ago • 0 comments

Through several rounds of changes we've made so far, the generic datasets (originally part of decaNLP) cannot be loaded properly in genienlp anymore and should be fixed. Also, many of those datasets (e.g. iwslt, sst, squad, imdb) are now part of datasets library. We should remove those obsolete implementations and associated metrics, and import them from datasets instead. We have added support to load new datasets from HF datasets. Check out https://github.com/stanford-oval/genienlp/blob/master/genienlp/tasks/hf_dataset.py for the implementation.

Mehrad0711 avatar Nov 30 '21 23:11 Mehrad0711