Sphereface-model icon indicating copy to clipboard operation
Sphereface-model copied to clipboard
verified publisher icon goodluckcwl

What is the relationship between lambda and batchsize?

Open double-vane opened this issue 8 years ago • 3 comments

How do you determine the lambda？

double-vane avatar Aug 08 '17 02:08 double-vane

Good question. Actually, the batch size I used is nothing special. I think the best lambda should be determined by experiment.

goodluckcwl avatar Aug 08 '17 03:08 goodluckcwl

@goodluckcwl Thank you very much. Does the lambda remain the same? layer { name: "fc6" type: "MarginInnerProduct" bottom: "fc5" bottom: "label" top: "fc6" top: "lambda" param { lr_mult: 1 decay_mult: 1 } margin_inner_product_param { num_output: 10575 type: QUADRUPLE weight_filler { type: "xavier" } base: 3.6 gamma: 0.12 power: 1 lambda_min: 3.6 iteration: 0 } }

double-vane avatar Aug 08 '17 05:08 double-vane

Note that this prototxt is used for fine-tune only. Besides, I don't known whether a larger batch size makes sense or not. You can have a try. For example, set batch_size=256.

goodluckcwl avatar Aug 08 '17 05:08 goodluckcwl