HiDDeN icon indicating copy to clipboard operation
HiDDeN copied to clipboard

Published 20 hours ago • verified publisher icon jirenz

Out of Memory!!

Open 0three opened this issue 5 years ago • 7 comments

I run this command in my lab servers. th main.lua --develop --name test-run --type float> And I got error like this. { maxPoolStride : 2 noProgress : false name : "test-run" learningRate : 0.001 transmissionJPEGU_yc : 5 batchSize : 12 develop : true optimType : "adam" adversaryFeatureDepth : 64 messageLength : 30 transmissionCropout : 0.4 transmissionDropout : 0.4 transmissionJPEGQuality : 50 type : "float" transmissionCropSize : 0.5 decoderConvolutions : 6 loadCheckpoint : "" fixImage : false encoderPreMessageConvolution : 3 noSave : false seed : 1234 maxPoolWindowSize : 4 transmissionGaussianSigma : 2 small : false encoderFeatureDepth : 64 confusionPer : 20 imageSize : 128 savePer : 20 imagePenaltyCoef : 1 testPer : 1 save : "checkpoints" transmissionJPEGU_yd : 0 fixMessage : false epochs : 200 decoderFeatureDepth : 64 transmissionNoiseType : "identity" thin : false transmissionJPEGCutoff : 5 transmissionJPEGU_uvd : 0 transmissionJPEGU_uvc : 3 small16 : false transmissionOutsize : 128 transmissionCombinedRecipe : "" adversary_gradient_scale : 0.1 adversaryConvolutions : 2 messagePenaltyCoef : 1 grayscale : false transmissionConcatenatedRecipe : "" encoderPostMessageConvolution : 1 randomImage : false } { beta1 : 0.9 epsilon : 1e-08 learningRateDecay : 0 learningRate : 0.001 beta2 : 0.999 } Loading training dataset Accepting non-grayscale input test-run: starting to train

epoch: 1

slurmstepd: error: Detected 1 oom-kill event(s) in step 10160.1 cgroup. Some of your processes may have been killed by the cgroup out-of-memory handler. srun: error: wmc-slave-g6: task 0: Out Of Memory

It looks like that I have no enough memory to run this. I just git clone these code and run the test command.

Could you plz share some requirements about this ? Thank you !

By the way, hope the pretrained models for research.

Thank you !

0three avatar Apr 29 '19 13:04 0three

My memory config.

cat /proc/meminfo MemTotal: 131663888 kB MemFree: 51571416 kB MemAvailable: 95205204 kB Buffers: 796904 kB Cached: 44341812 kB SwapCached: 2428 kB Active: 62199800 kB Inactive: 14218780 kB Active(anon): 30307824 kB Inactive(anon): 3407284 kB Active(file): 31891976 kB Inactive(file): 10811496 kB Unevictable: 0 kB Mlocked: 0 kB SwapTotal: 999420 kB SwapFree: 949476 kB Dirty: 148 kB Writeback: 0 kB AnonPages: 31277964 kB Mapped: 2979180 kB Shmem: 2435292 kB Slab: 1860776 kB SReclaimable: 1556396 kB SUnreclaim: 304380 kB KernelStack: 14688 kB PageTables: 94932 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 66831364 kB Committed_AS: 36091084 kB VmallocTotal: 34359738367 kB VmallocUsed: 0 kB VmallocChunk: 0 kB HardwareCorrupted: 0 kB AnonHugePages: 0 kB CmaTotal: 0 kB CmaFree: 0 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB DirectMap4k: 34013280 kB DirectMap2M: 96712704 kB DirectMap1G: 5242880 kB

0three avatar Apr 29 '19 13:04 0three

srun nvidia-smi Mon Apr 29 21:08:18 2019 +-----------------------------------------------------------------------------+ | NVIDIA-SMI 410.72 Driver Version: 410.72 CUDA Version: 10.0 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | |===============================+======================+======================| | 0 GeForce GTX 108... Off | 00000000:B2:00.0 Off | N/A | | 23% 26C P8 15W / 250W | 0MiB / 11178MiB | 0% Default | +-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+ | Processes: GPU Memory | | GPU PID Type Process name Usage | |=============================================================================| | No running processes found | +-----------------------------------------------------------------------------+

0three avatar Apr 29 '19 13:04 0three

What you have should be enough, or just a bit short. Can you try --batch-size 10?

jirenz avatar May 04 '19 19:05 jirenz

I have the following problems. Would you mind helping me solve them lua.exe: main.lua:2: module 'image' not found: no field package.preload['image'] no file '.\image.lua' no file 'G:\Program Files (x86)\Lua\5.1\lua\image.lua' no file 'G:\Program Files (x86)\Lua\5.1\lua\image\init.lua' no file 'G:\Program Files (x86)\Lua\5.1\image.lua' no file 'G:\Program Files (x86)\Lua\5.1\image\init.lua' no file 'G:\Program Files (x86)\Lua\5.1\lua\image.luac' no file '.\image.dll' no file 'G:\Program Files (x86)\Lua\5.1\image.dll' no file 'G:\Program Files (x86)\Lua\5.1\loadall.dll' no file 'G:\Program Files (x86)\Lua\5.1\clibs\image.dll' no file 'G:\Program Files (x86)\Lua\5.1\clibs\loadall.dll' no file '.\image51.dll' no file 'G:\Program Files (x86)\Lua\5.1\image51.dll' no file 'G:\Program Files (x86)\Lua\5.1\clibs\image51.dll' stack traceback: [C]: in function 'require' main.lua:2: in main chunk [C]: ?

HarryAB avatar Dec 05 '19 08:12 HarryAB

I have the following problems. Would you mind helping me solve them lua.exe: main.lua:2: module 'image' not found: no field package.preload['image'] no file '.\image.lua' no file 'G:\Program Files (x86)\Lua\5.1\lua\image.lua' no file 'G:\Program Files (x86)\Lua\5.1\lua\image\init.lua' no file 'G:\Program Files (x86)\Lua\5.1\image.lua' no file 'G:\Program Files (x86)\Lua\5.1\image\init.lua' no file 'G:\Program Files (x86)\Lua\5.1\lua\image.luac' no file '.\image.dll' no file 'G:\Program Files (x86)\Lua\5.1\image.dll' no file 'G:\Program Files (x86)\Lua\5.1\loadall.dll' no file 'G:\Program Files (x86)\Lua\5.1\clibs\image.dll' no file 'G:\Program Files (x86)\Lua\5.1\clibs\loadall.dll' no file '.\image51.dll' no file 'G:\Program Files (x86)\Lua\5.1\image51.dll' no file 'G:\Program Files (x86)\Lua\5.1\clibs\image51.dll' stack traceback: [C]: in function 'require' main.lua:2: in main chunk [C]: ?

I meet the same problem,Do you solve it?

neverwinHao avatar May 07 '24 12:05 neverwinHao

You probably don't have Torch installed. https://stackoverflow.com/questions/48277220/get-lua-running-with-torch-on-windows-10-with-limited-admin-rights

I have the following problems. Would you mind helping me solve them lua.exe: main.lua:2: module 'image' not found: no field package.preload['image'] no file '.\image.lua' no file 'G:\Program Files (x86)\Lua\5.1\lua\image.lua' no file 'G:\Program Files (x86)\Lua\5.1\lua\image\init.lua' no file 'G:\Program Files (x86)\Lua\5.1\image.lua' no file 'G:\Program Files (x86)\Lua\5.1\image\init.lua' no file 'G:\Program Files (x86)\Lua\5.1\lua\image.luac' no file '.\image.dll' no file 'G:\Program Files (x86)\Lua\5.1\image.dll' no file 'G:\Program Files (x86)\Lua\5.1\loadall.dll' no file 'G:\Program Files (x86)\Lua\5.1\clibs\image.dll' no file 'G:\Program Files (x86)\Lua\5.1\clibs\loadall.dll' no file '.\image51.dll' no file 'G:\Program Files (x86)\Lua\5.1\image51.dll' no file 'G:\Program Files (x86)\Lua\5.1\clibs\image51.dll' stack traceback: [C]: in function 'require' main.lua:2: in main chunk [C]: ?

I meet the same problem,Do you solve it?

epachamo avatar May 07 '24 19:05 epachamo

I meet the same problem,Do you solve it?

hello, i think we can discuss how to solve it!

neverwinHao avatar May 08 '24 10:05 neverwinHao