keras-cv icon indicating copy to clipboard operation
keras-cv copied to clipboard

Redundant bias paramter in ResNet's conv2d.

Open sebastian-sz opened this issue 2 years ago • 1 comments

Short description In tf.keras by default Conv2D has bias enabled. We should explicitly set use_bias=False if the convolution layer is followed by BatchNormalization.

This is the done in some existing Keras-CV implementations:

  • DarkNet Conv Block - link
  • DenseNet - link
  • ResNet V2 - link

But not in resnet_v1 - all ResNet's have bias enabled, which causes redundant computation and is not aligned with other implementations.

I think this is rather a quick fix - add use_bias=False parameter to the Conv2d's in the Block function. If you approve I could submit a PR.

sebastian-sz avatar Jul 29 '22 10:07 sebastian-sz

Gently pinging @LukeWood for opinion / green light here.

sebastian-sz avatar Aug 16 '22 05:08 sebastian-sz

Interesting, could you send a PR so we can verify this? I think you're correct, but I would like to see the updated code.

LukeWood avatar Aug 16 '22 19:08 LukeWood

@LukeWood yes, submitted the PR here https://github.com/keras-team/keras-cv/pull/697

sebastian-sz avatar Aug 17 '22 13:08 sebastian-sz