YouTokenToMe icon indicating copy to clipboard operation
YouTokenToMe copied to clipboard

Published 20 hours ago • verified publisher icon VKCOM

Process killed?

Open miguelvictor opened this issue 3 years ago • 4 comments

Any idea why this would happen?

Training parameters
  input: _training_aux
  model: vocab.bpe
  vocab_size: 24000
  n_threads: 8
  character_coverage: 1
  pad: 0
  unk: 1
  bos: 2
  eos: 3

reading file...
learning bpe...
number of unique characters in the training data: 224
number of deleted characters: 0
number of unique characters left: 224
Killed

OS: Pop!_OS 20.04 LTS Version: 1.0.6 Input file size: 1.8G

miguelvictor avatar Jul 31 '20 16:07 miguelvictor

How much RAM do you have?

xbelonogov avatar Jul 31 '20 17:07 xbelonogov

@xbelonogov 8GB (swap is 4GB)

miguelvictor avatar Aug 01 '20 07:08 miguelvictor

The problem was my RAM is not enough. I tried running it on my 16GB macbook pro and it worked fine. It used about 22.6 GB of RAM.

@xbelonogov One suggestion though, would it be possible to compute the approximate ram usage and warn users if stuff like this will happen?

miguelvictor avatar Aug 01 '20 08:08 miguelvictor

Yes, that looks reasonable.

xbelonogov avatar Aug 01 '20 08:08 xbelonogov