Convolutional Block Attention Module

[innat]

https://arxiv.org/abs/1807.06521 Cited by 4701

[bhack]

https://github.com/tensorflow/models/blob/master/official/vision/beta/projects/yolo/modeling/layers/nn_blocks.py#L1333

[LukeWood]

Yep, highly cited and used. makes sense; perhaps as a part of the upoming training templates effort.

[old-school-kid]

Hi @LukeWood @innat I would like to work on this! I was thinking of something like this

def attach_attention_module(net, attention_module):
  if attention_module == 'se_block': # SE_block
    net = se_block(net)
  elif attention_module == 'bam_block': # BAM_block
    net = cbam_block(net)
  elif attention_module == 'cbam_block': # CBAM_block
    net = cbam_block(net)
  elif attention_module == 'gala_block': # GALA_block
    net = cbam_block(net)
  else:
    raise Exception(f"{attention_module} is not supported attention module.")
  return net

We can also give option to pass their own attention module. Thoghts?

[innat]

@old-school-kid That would be cool. But I think according to keras design, it might want to include a separate class for each.. (SE block in currently in progress https://github.com/keras-team/keras-cv/pull/505).

[bhack]

Now at https://github.com/tensorflow/models/blob/master/official/vision/modeling/layers/nn_blocks.py

[LukeWood]

Thanks for your enthusiasm @GLOMQuyet !

Apologies for shifting priorities and changes in terms of what we are interested in merging. We've done a lot of work to figure out exactly what our value add to users is, what components we want to offer as APIs, and we don't want to offer as APIs in the last few years. For now, I think closing this PR makes sense and if models need this component we can implement this style layer alongside the model.

This usually has to happen to match exact numerics of architectures anyways. For now lets close, and if theres a super strong user need to fill we can re-open.