PyTorch-NLP

PyTorch-NLP copied to clipboard

Add GLUE datasets

7

GLUE datasets are standard for evaluating NLU tasks. > In pursuit of this objective, we introduce the General Language Understanding Evaluation benchmark > (GLUE), a tool for evaluating and analyzing...

PetrochukM

Wrong number of classes is derived from `label_encoder.vocab_size`

5

## Behaviors The following code snippet is directly taken from `README.md` of the this library (see [here](https://github.com/PetrochukM/PyTorch-NLP/blob/master/README.md)). I am expecting the following `n_class` to be equal to 2 (i.e. there...

guanqun-yang

MaxTokensBatchSampler

1

Is there any straightforward way to specify the maximum number of tokens per batch in a sampler (eg.: BucketBatchSampler)? Reducing the amount of padding per batch is critical for performance...

salvacarrion

Added Rogue Metric

1

Cleared an existing issue, added the rouge metric - rouge_n and rouge_l

PriyaDeshpande1605

Write blog post

1

## Expected Behavior PyTorch-NLP might be hard to understand if you are starting a NLP project, I expect a blog post to exist to help fresh faces. Example links: https://hn.algolia.com/?query=nlp&sort=byPopularity&prefix&page=0&dateRange=all&type=story...

PetrochukM

Hi, I have multiple large-scale datasets in TFDS format, which needs to be converted to iterative datasets, and I want to trani large-scale T5 model on TPUs with them, for...

rabeehk

`fork_rng_wrap` does not propagate function arguments `fork_rng_wrap`, and it should.

PetrochukM

Simplify `Encoder`: Special Tokens, OOB, Batch Encoding

1

The `Encoder` is unnecessarily complex: - It has OOB that's hard to reason with - The `Encoder` implementation is coupled with special tokens - The `batch_encoding` isn't helpful We could...

PetrochukM

Support loading fasttext model from custom file

7

Add two simple parameters to class FastText, making it possible to load fasttext model from custom file.

YuhengHuang42

ONNX support for Encoders

1

Based on some initial experiments, it looks like the Encoder classes can not be exported to the ONNX format using `torch.onnx.export`. It would be great to be able to package...

cjermain