text
text copied to clipboard
pytorch
Models, data loaders and abstractions for language processing, powered by PyTorch
Add warehouse double-conversion to support loongarch, which has been tested on loongarch and can be passed
When attempting to use the IMDB api, I got results that were different from what the docs suggested. This PR attempts to update the docs with the correct output of...
Summary: ## Problem: pytext got "No module named 'pytorch'" in issue https://github.com/facebookresearch/pytext/issues/1706 It's due to `from pytorch.text.fb.utils import PATH_MANAGER` is internal only but imported in pytext. Actually, `pytorch/text/fb/utils/__init__.py` should be...
## 🐛 Bug IWSLT datasets are not properly unpacked from the downloaded `tgz` file when using `torchtext.datasets.IWSLT*`. When I unpack the nested `tgz` files by hand everything works as expected....
## 🐛 Bug The `test_vocab_from_raw_text_file` test is failing on CI for linux platforms due to segmantation faults (see [sample CI job](https://app.circleci.com/pipelines/github/pytorch/text/6713/workflows/016984af-210f-4ec1-ac23-e7b31ea6f465/jobs/231623)). We currently disable the test on Linux to unblock...
## 📚 Documentation **Description** In the example shown, a `test.csv` file is used as an input. However, it is unclear whether the entire row is considered as a single sentence...
## 🚀 Feature There should be a `verbose` parameter, e.g. ```python torchtext.data.functional.generate_sp_model(..., verbose=0) ``` **Motivation** At the moment, there's no documented way to change the verbosity, but a lot of...
old: ```python def stratify(examples, strata_field): unique_strata = set(getattr(example, strata_field)) strata_maps = {s: [] for s in unique_strata} for example in examples: strata_maps[getattr(example, strata_field)].append(example) return list(strata_maps.values()) ``` new: ```python def stratify(examples,...
Adds the ability to assign pretrained embeddings as a "fallback" (motivated by https://github.com/pytorch/text/issues/446). This is my first PR, so let me know if anything is suboptimal. Perhaps it would be...
