word-embedding-dimensionality-selection
word-embedding-dimensionality-selection copied to clipboard
ziyin-dl
core dumped error
error message:
Segmentation fault (core dumped) nohup python -m main --file data/train.txt --config_file config/train.yml --algorithm word2vec
demo data can run successfully in same environment.
My config file is:
skip_window: 5 neg_samples: 1 vocabulary_size: 94418 min_count: 1
My train_data: 539 M txt file. system: Ubuntu 14.04 hardware: 50 cores and 256GB RAM
That might be caused by the failure of some linear algebra subroutines. But hard to say without further information. What's the matrix size you are using and what is your RAM size?
On Wed, Nov 20, 2019, 9:11 PM SXH [email protected] wrote:
Program has run two days before Segmentation fault.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ziyin-dl/word-embedding-dimensionality-selection/issues/18?email_source=notifications&email_token=AB7IJUGOUYEYKAWNJYHKIR3QUXU3XA5CNFSM4JP32R7KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEEYLOWY#issuecomment-556840795, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB7IJUEY3T4OZRCN7LS253TQUXU3XANCNFSM4JP32R7A .
What was the RAM usage when it occurred?
On Thu, Nov 21, 2019, 7:16 PM Zi Yin [email protected] wrote:
That might be caused by the failure of some linear algebra subroutines. But hard to say without further information. What's the matrix size you are using and what is your RAM size?
On Wed, Nov 20, 2019, 9:11 PM SXH [email protected] wrote:
Program has run two days before Segmentation fault.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ziyin-dl/word-embedding-dimensionality-selection/issues/18?email_source=notifications&email_token=AB7IJUGOUYEYKAWNJYHKIR3QUXU3XA5CNFSM4JP32R7KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEEYLOWY#issuecomment-556840795, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB7IJUEY3T4OZRCN7LS253TQUXU3XANCNFSM4JP32R7A .
What was the RAM usage when it occurred? … On Thu, Nov 21, 2019, 7:16 PM Zi Yin @.> wrote: That might be caused by the failure of some linear algebra subroutines. But hard to say without further information. What's the matrix size you are using and what is your RAM size? On Wed, Nov 20, 2019, 9:11 PM SXH @.> wrote: > Program has run two days before Segmentation fault. > > — > You are receiving this because you are subscribed to this thread. > Reply to this email directly, view it on GitHub > <#18?email_source=notifications&email_token=AB7IJUGOUYEYKAWNJYHKIR3QUXU3XA5CNFSM4JP32R7KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEEYLOWY#issuecomment-556840795>, > or unsubscribe > https://github.com/notifications/unsubscribe-auth/AB7IJUEY3T4OZRCN7LS253TQUXU3XANCNFSM4JP32R7A > . >
Thanks for reply, I think the usage of RAM has two stags. The first stage used around 240 GB RAM, after that, the two stage used around 180GB RAM, then the program raise segment fault.