deep-code-search icon indicating copy to clipboard operation
deep-code-search copied to clipboard

Published 20 hours ago • verified publisher icon guxd

How to convert .txt file to .h5 file

Open XiaoXiaoYi123 opened this issue 2 years ago • 4 comments

ese /indices and /phrases?

XiaoXiaoYi123 avatar Mar 08 '22 02:03 XiaoXiaoYi123

请问可以提供将txt文本序列转为对应的.h5文件的实现代码吗？

li-car-fei avatar May 13 '22 14:05 li-car-fei

@li-car-fei https://github.com/guxd/DialogBERT/blob/master/prepare_data.py 参考这个，用pytable 把dialog (a list of sequences)转成earray.

guxd avatar May 13 '22 14:05 guxd

@li-car-fei https://github.com/guxd/DialogBERT/blob/master/prepare_data.py 参考这个，用pytable 把dialog (a list of sequences)转成earray.

在deepcs中，你是使用了哪里的tokenizer呢？

li-car-fei avatar May 14 '22 02:05 li-car-fei

deepcs没有用第三方tokenizer,是自己实现的简易的tokenizer. 直接把字符转换成字典里的序号

guxd avatar May 14 '22 07:05 guxd