F0209 17:45:34.604908 6609 syncedmem.cpp:51] Check failed: error == cudaSuccess (2 vs. 0) out of memory

[martende]

on testing caffe-segnet on /SegNet/Models/segnet_basic_solver.prototxt I am always receiving out of memory. My card is:

01:00.0 VGA compatible controller: NVIDIA Corporation GM107 [GeForce GTX 745] (rev a2) (prog-if 00 [VGA controller])
	Subsystem: Hewlett-Packard Company Device 6893
	Flags: bus master, fast devsel, latency 0, IRQ 34
	Memory at f6000000 (32-bit, non-prefetchable) [size=16M]
	Memory at e0000000 (64-bit, prefetchable) [size=256M]
	Memory at f0000000 (64-bit, prefetchable) [size=32M]
	I/O ports at e000 [size=128]
	[virtual] Expansion ROM at f7000000 [disabled] [size=512K]
	Capabilities: [60] Power Management version 3
	Capabilities: [68] MSI: Enable+ Count=1/1 Maskable- 64bit+
	Capabilities: [78] Express Legacy Endpoint, MSI 00
	Capabilities: [100] Virtual Channel
	Capabilities: [250] Latency Tolerance Reporting
	Capabilities: [258] L1 PM Substates
	Capabilities: [128] Power Budgeting <?>
	Capabilities: [600] Vendor Specific Information: ID=0001 Rev=1 Len=024 <?>
	Capabilities: [900] #19
	Kernel driver in use: nvidia

[beejisbrigit]

@martende try reducing your batch size from 6 (the default I believe) to something else; you may have to try multiple values. I had this problem initially using a Titan X GPU with ~12 GB of memory and reduced my batch size to 4. After this, no problem.

good luck!

[yifaliu]

@martende This may be related to the cuDNN version. You can degrade the cuDNN version or comment the line of USE_CUDNN := 1 in Makefile.config