Tien Tr

Need to upgrade `tensorflow` and update deprecated methods as shown in pytorch lighning 1.7.0. See my pull request

You can, however GPT intend to bias toward to last chunk of your dataset. To avoid that, you can use `merge_datasets()`.

Try set `num_steps` and `batch_size` higher

That's because `NamedTemporaryFile` will delete the temp folder before `faiss` access. For more infor in [python documentation](url). To fix this, simply pass `delete=False` to `NamedTemporaryFile` parameters in `get_index_size` func of...

@rom1504 that because `NamedTemporaryFile` of python (read more [here](https://docs.python.org/3/library/tempfile.html) ). `Whether the name can be used to open the file a second time, while the named temporary file is still...

When training loss equal to 0 the model will remember all information in dataset so input prompt should be different from dataset. A large dataset is important and necessary to...

Hello, I confirmed this issue is persisted. My setup GPT-2 with Zero3 Using custom dataset Config: ```from colossalai.nn.optimizer import HybridAdam from colossalai.zero.shard_utils import TensorShardStrategy from titans.model.gpt import gpt2_large from colossalai.amp...

In order to get PredPatt works, you will need - JPype 0.7.4 - Python 3.6.2 - Download Parser, Grammar, CoreNLP manually -> Set fixed downloading location in `UDParser.py` - Add...