Segmentation fault due to higher batch size

[rohitqut]

Hello,

For creating the snapshot (through solver.prototxt), I need to provide batch size more than 10,000 in numbers inside c3d_sport1m_feature_extractor_frm.prototxt but this is giving segmentation fault error in the output. Why this is happening and what is the possible solution?

[dutran]

I don't really understand the issue here. If it run out of memory, you can just simple reduce the batch-size. Also 10,000 is a very large batch size.

[rohitqut]

OK I followed your advice and reduced the batch size number for creating the snapshot. Now, I move on to extract the features. 2 arguments as mentioned in the C3D user guide while extracting features are and . Here again, same segmentation fault error comes up when I give higher values like <10,000> <100>. Since I am dealing with very smaller patches, therefore, total feature extractions are also higher. Also, I observed that if is decreased and are increased, say like this <4000> <300> then features does not get extracted in synchronized manner at all (synchronize I mean to say sequentially folder by folder). Some random extractions happens in some folders and many other folders does not contains any extractions.

Really not getting how to tackle with and if my requirement is to extract much higher number of features for the smaller patches in different sets of folders.

[rohitqut]

2 arguments as mentioned in the C3D user guide while extracting features are mini batch size and number of mini batches.

Also, I observed that if mini batch size is decreased and number of mini batches are increased

Really not getting how to tackle with mini batch size and number of mini batches.

Sorry I copied directly from C3D user guide that is why I think these two argument names did not appear.

[sainisanjay]

@rohitqut you have to decide batch size according to you GPU memory size,,,as @dutran mentioned in the instruction... if you have 6GB GPU: use batch_size of ~30. 12GB: ~50. Adjust for your GPU memory.

[dutran]

@sainisanjay thanks for helping!