Xin Zhang

Hi, I think the chunking is still needed. Recall that all docs are gatherd and encoded with [`SentenceTransformer.encode`](https://github.com/UKPLab/sentence-transformers/blob/master/sentence_transformers/SentenceTransformer.py#L195) at once, where `torch.stack(all_embeddings)` is called to merge batched embeddings. Since `stack`...

And I write a simulation script that can quickly reproduct the case. ``` import subprocess import tqdm import torch command = 'nvidia-smi' embedding_dim = 1024 batch_size = 128 print('model and...

This is indeed a problem with sentence-transformers, as they did not consider calls at the million level. I agree that your method is a reasonable solution. I think the large...

感谢提问。单纯的推理功能正在开发，预计半个月到一个月，目前有 [test](https://github.com/modelscope/AdaSeq/blob/master/adaseq/commands/test.py) 功能。 这个文件是 [`ner_dumpper`](https://github.com/modelscope/AdaSeq/blob/master/adaseq/data/dataset_dumpers/named_entity_recognition_dataset_dumper.py) 保存的，它就是一个在评测调用的特殊的 `metric`。

您好，之前钉钉群里也有人提到了这个问题，可能是相同原因，麻烦可以尝试一下呢。 很抱歉最近开发者没有及时跟进issues。   可以尝试先把modelscope版本降级到1.4.2以下，运行pip install modelscope==1.4.2。 这个问题是ms版本升级导致的兼容性问题，我们会尽快修复！

您好，可能是文本风格的原因。检测器其实是根据行文的风格和特征来判断是否是chatgpt生成的，所以如果风格相似，确实会误判 😄

@NJRBarry

Hi, Generally, If the source datasets used in this corpus has a specific license, our products follow the same. If not, or if the datasets are crawled by ourselves, they...

But the license may change, we will decide it ASAP.

We use CC-BY-SA. [dataset copyright](https://github.com/Hello-SimpleAI/chatgpt-comparison-detection#dataset-copyright)