Stacked_Hourglass_Network_Keras
Stacked_Hourglass_Network_Keras copied to clipboard
yuanyuanli85
batch size 2
Can you explain why you have such a small batch size in the training command please? thanks
The provided commands just show how to use the 'train.py'. Actually, i used batch_size 24 in my expriments. Thank you for pointing it out. I will made the change on readme to avoid confusing.
When I use batch_size of 10 I already get an out of memory error (on a GOU with 8 GB). Can you explain why each sample takes up so much space? I can't figure it out, it seems to me that each sample is only 1*inres+ (14 *n_stacks)*outres big. Or is there something wrong with my reasoning.
On Mon, Oct 22, 2018 at 3:02 AM VictorLi [email protected] wrote:
The provided commands just show how to use the 'train.py'. Actually, i used batch_size 24 in my expriments. Thank you for pointing it out. I will made the change on readme to avoid confusing.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/yuanyuanli85/Stacked_Hourglass_Network_Keras/issues/7#issuecomment-431719935, or mute the thread https://github.com/notifications/unsubscribe-auth/AG5qEA2dW4ql7-IzguH3uQuqUV_bFx5-ks5unRkjgaJpZM4XyaGB .
The training took much more memory than the input & output memory due to the forward/backward of each layer. If you want to avoid out-of-memory issue, you can try to decrease inres or stack number or batch_size.
"due to the forward/backward of each layer" what do you mean by "layer" here, is it 1 hourglass stack or one bottleneck block or something else? Because I have trained other networks with 100's of images in a batch (not in keras though, I used pytorch). Is there a way to print the memory size in megabytes of a batch? Thanks!
On Tue, Oct 23, 2018 at 2:04 AM VictorLi [email protected] wrote:
The training took much more memory than the input & output memory due to the forward/backward of each layer. If you want to avoid out-of-memory issue, you can try to decrease inres or stack number or batch_size.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/yuanyuanli85/Stacked_Hourglass_Network_Keras/issues/7#issuecomment-432034442, or mute the thread https://github.com/notifications/unsubscribe-auth/AG5qEFyTmPKLa71E3uWLTiA-Tkzsra9pks5unl0mgaJpZM4XyaGB .
Also, in the validation callback here,
https://github.com/yuanyuanli85/Stacked_Hourglass_Network_Keras/blob/eddf0ae15715a88d7859847cfff5f5092b260ae1/src/eval/eval_callback.py#L29
Shouldn't it be valdata.generator(batch_size, **2**, sigma=2, is_shuffle=False, with_meta=True):
Doesn't 8 mean the numstacks are being set incorrectly.
@pavan4 In eval_callback.py, the number of stackes in generator does not matter because the _gthmap not used during validation. Instead, we use the ground truth infor in _meta to get score.