Possible Errors in Current Version - some variables are not trained

[XiaoYunZhou27]

Hi,

Thank you for contributing this fantastic code. It helps me a lot.

However, as I am learning deeper and deeper, I found that the "wd, bd, weight, and bias" in the function "create_conv_net" is not appended into the trainable variable - "variables". I make these variables trainable and the result improves a lot.

Could you please kindly discuss and tell me whether my concern makes sense, please? Thanks a lot.

[jakeret]

Hi thanks for this input. Which variables do mean exactly, could you point me to the relevant code line?

The variables list is only used when regularization is used. Therefore I would assume that it should only affect the generalization error. Do you see a difference during training?

[XiaoYunZhou27]

Hi,

Thanks for the replying.

Could you please kindly go to function "def create_conv_net" - "up layers" - wd and bd are not appended into the "variables", also "def create_conv_net" - "Output Map" - weight and bias are not appended into the "variables".

I think the "variables" is the variable which will be updated by the SGD. If I misunderstood, please let me know. I am learning.

Thank you very much for this contribution.

[jakeret]

Ok is see. Yes you're correct. Thank you for pointing this out. Those variables should be appended to the list. Weights and biases are indeed the ones updated by SGD. However, this happens internally in tensorflow and does not depend on the variables list. Therefore, training should not be affected.

If you have time, it would be great if you could fix the bug and send me a git Pull Request.

[meijie0401]

So even if you don't append 'wd, bd, weight, and bias' to varilables list, they are still can be updated by tensorflow ? Only when we want to use regularization, we should append them to the varilables list. Am I right?

[jakeret]

Yes. However I don't know how big the effect is if they are in the list or not