Santosh Gupta

I am looking at section 4.2 of the BERT paper or how to set up BERT for reading comprehension. It looks like a module needs to be added to the...

Pre-trained model ?

1

Hello, I was wondering if a pre-trained model / weights will be released for this?

https://github.com/WING-NUS/scisumm-corpus This is a computational research papers summarization competition.

Difference between highlight and selection not apparent for editor focus. Is this from the theme or VScode?

1

 the 'cache' at the top is selected, the one at the bottom is highlighted because all instances of 'cache' are highlighted when one is selected. The difference between highlighted...

How to use for word2vec training?

9

Sorry for the newbie question, but I'm having a bit of trouble in trying to use the library. Specifically what I'm trying to do is embedding training, like word2vec. So...

How were the users chosen?

2

I see that there are 876,145 total users in the dataset, but goodreads has 90 million users (as of july 2019). I was wondering how were those 876,145 users selected....

I'm curious about how PICO and dependency parsing was trained using sciBert. For PICO, I can imagine training being set up like squad, where a 'question' is one if the...

How to diagnose the cause of frozen training.

6

I was running some training when the training seemed to freeze in the middle of an epoch. I don't see any other issues for this, so it could be related...

Deepspeed model engine can not load a saved checkpoint. "RuntimeError: The size of tensor a x must match the size of tensor b y at non-singleton dimension z"

1

During training, I would periodically save a checkpoint using `model_engine.save_checkpoint` However, `model_engine.load_checkpoint` is resulting in this output ``` [2021-07-08 19:55:42,454] [INFO] [state_dict_factory.py:165:check_ckpt_list] checkpoint file list: ['/home/santosh/deepspeed_checkpoints/secondTest/global_step18825/zero_pp_rank_0_mp_rank_00_model_states.pt'] [2021-07-08 19:55:42,468] [INFO] [state_dict_factory.py:55:load]...

When I run Test, the resulting file has 10-15% less samples than the number of samples in the input (number of each samples in each file created by `format_to_bert`). I...