Shoku Yanagawa

export OPENAI_GPT2_CHECKPOINT_PATH=./model/ckpt-301 transformers-cli convert --model_type gpt2 \ > --tf_checkpoint $OPENAI_GPT2_CHECKPOINT_PATH \ > --pytorch_dump_output ./pytorch_dump_output/pytorch_model.bin \ > --config ./model/model_par.json \ > INFO:transformers.modeling_gpt2:Converting TensorFlow checkpoint from /home/mycat/gpt-2-tensorflow2.0/model/ckpt-301 INFO:transformers.modeling_gpt2:Loading TF weight _CHECKPOINTABLE_OBJECT_GRAPH with...

How to generate or convert vocab.json, merges.txt, and config.json to match huggingface/transformers requirements ?

4

Hi, Great work !! I am migrating to GPT2 japanese version for pytorch Pretrained GPT2 japanese is working fine. Thanks a lot. However, I miss vocab.json, merges.txt and config.json files...

Some example build fails on Arm64 environment(Armbian Focal) with Libtorch1.10.1 source build

2

I am using Rockpro64 SBC with Armbian Focal and I am developing Rust application. Libtorch1.10.1 native build for Arm64 from pytorch source tree is successful. I followed the tch-rs instruction...

vicuna-13b v1.1 Strange answer

2

After an answer in Japanese, strange English message is followed. Name: fschat Version: 0.2.1 python3 -m fastchat.serve.cli --style rich --model-path /path/to/vicuna-13b-v1.1 --load-8bit USER: こんにちわ、雪国の温泉宿に泊まりたいのですが、何処の温泉宿がお勧めですか。 ASSISTANT: どういたしましょうか？お勧めできる雪国の温泉宿は以下の通りです。 1 北陸の温泉宿「タンドー」 北陸市にある温泉宿で、美味しい食事やお風呂の入り口を楽しめます。 2...