text

text copied to clipboard

fix text_classfication dataset import package

3

change torchtext.experimental.datasets to torchtext.datasets for latest version.

wenjun93

This PR adds Windows build instructions for the repo. The repo does not compile without using the [MSVC x64 toolset](https://docs.microsoft.com/en-us/cpp/build/how-to-enable-a-64-bit-visual-cpp-toolset-on-the-command-line?view=msvc-160#use-vcvarsallbat-to-set-a-64-bit-hosted-build-architecture). The given instructions in the PR is the same way...

mstfbl

[WIP] using std::array in vocab for additional speed-ups

1

We replace container type of stoi_ from std::vector to std::array. This bring in slight additional improvements in look-up speed

parmeet

Helper function for Vocab object

5

I found it hard to understand how to preform a "full loop" of Token -> index -> Token, I think the intended manner is to just access the itos variable...

TuckerBMorgan

Doc string for the examples of datasets

1

Add the example doc strings to `torchtext.datasets`. 

zhangguanheng66

For FB internal test, the `raw_datasets.json` contents are not valid. Update the format to pass the internal lint check.

zhangguanheng66

Currently `torchtext` runs tests by `pytest test`. This is fine for development but one cannot run unit test on installed package in this manner because running test from the root...

mthrok

Add Trec dataset

1

The legacy Trec dataset was retired in `torchtext.legacy` folder. This one yields the raw text strings.

zhangguanheng66

Add Stanford Sentiment Treebank (SST) dataset

1

The legacy SST was retired in `torchtext.legacy` folder. This one yields the raw text strings.

zhangguanheng66

Add Natural Language Inference datasets

1

The following three datasets have been retired in the `legacy.datasets` folder. We are re-writing these by yielding the raw texts: - SNLI - MatchedMultiNLI ([link](https://www.kaggle.com/c/multinli-matched-open-evaluation)) - MismatchedMultiNLI ([link](https://www.kaggle.com/c/multinli-mismatched-open-evaluation)) Unfortunately, The...

zhangguanheng66