easy-lm-trainer icon indicating copy to clipboard operation
easy-lm-trainer copied to clipboard

verified publisher icon Beomi
β†’

Metadata

πŸ€— μ΅œμ†Œν•œμ˜ μ„ΈνŒ…μœΌλ‘œ LM을 ν•™μŠ΅ν•˜κΈ° μœ„ν•œ μƒ˜ν”Œμ½”λ“œ

Results 2 easy-lm-trainer issues
Sort by recently updated
recently updated
newest added

main.py 문의

park1200656 avatar park1200656

λͺ¨λΈ μ €μž₯ 이슈

5 comment

μ•ˆλ…•ν•˜μ„Έμš”. μ˜¬λ €μ£Όμ‹  μ½”λ“œλ‘œ λͺ¨λΈμ„ ν•™μŠ΅ν•˜κ³  μ €μž₯ν•˜λ˜ 도쀑, μ €μž₯ κ΄€λ ¨ λ¬Έμ˜κ°€ μžˆμ–΄ 이슈 λ‚¨κΉλ‹ˆλ‹€. μ €λŠ” A100 40gb 8μž₯을 μ‚¬μš©ν•˜κ³ , 그에 μ•„λž˜μ²˜λŸΌ num_procλ₯Ό 8둜 μ…‹νŒ…μ„ ν•˜μ˜€μŠ΅λ‹ˆλ‹€. tokenized_datasets = datasets.map( tokenize_function, batched=True, num_proc=8,...

park1200656 avatar park1200656

← Metadata

57
Stars
9
Forks
Watchers

Owner

verified publisher icon Beomi

Metadata

πŸ€— μ΅œμ†Œν•œμ˜ μ„ΈνŒ…μœΌλ‘œ LM을 ν•™μŠ΅ν•˜κΈ° μœ„ν•œ μƒ˜ν”Œμ½”λ“œ

Back