keras-cv icon indicating copy to clipboard operation
keras-cv copied to clipboard

Add MAXIM model

Open innat opened this issue 3 years ago • 9 comments

Short Description

MAXIM: Multi-Axis MLP for Image Processing: I think it's a follow-up work of MaxViT from google. It shows a great performance on the following low-vision task, i.e. for OCR.

image3

It looks useful for real world problem (blurry, haze, rain-drop). Such problems are very acute for vision project. So, such solutions might weight quick adaptation of kcv in industrial projects (alike).

Papers

  • https://arxiv.org/pdf/2201.02973.pdf image

Existing Implementations

  • In Jax: https://github.com/google-research/maxim
  • maxim-tf
  • Google-blog: https://ai.googleblog.com/2022/09/a-multi-axis-approach-for-vision.html
  • pytorch: https://github.com/vztu/maxim-pytorch (WIP).

cc. @vztu

innat avatar Oct 14 '22 18:10 innat

Open this up for contribution

tanzhenyu avatar Oct 27 '22 16:10 tanzhenyu

cc @sayakpaul

innat avatar Oct 28 '22 03:10 innat

I'm working on integrating the maxim model and referencing the original flax implementation. I have one question: do we need weights as well? @tanzhenyu

IMvision12 avatar Dec 28 '22 11:12 IMvision12

I'm working on integrating the maxim model and referencing the original flax implementation. I have one question: do we need weights as well? @tanzhenyu

Yes, we need the weights. The development cycle would be similar to ViT: https://github.com/keras-team/keras-cv/pull/1014, add the model and port the weights, also add the conversion scripts for porting the weights

tanzhenyu avatar Dec 28 '22 13:12 tanzhenyu

Yes, we need the weights. The development cycle would be similar to ViT: https://github.com/keras-team/keras-cv/pull/1014, add the model and port the weights, also add the conversion scripts for porting the weights

I hope we could standardize a bit the model contribution process after we will handle:

https://github.com/keras-team/keras-cv/discussions/954

bhack avatar Dec 28 '22 14:12 bhack

I'm working on integrating the maxim model and referencing the original flax implementation. I have one question: do we need weights as well? @tanzhenyu

Yes, we need the weights. The development cycle would be similar to ViT: #1014, add the model and port the weights, also add the conversion scripts for porting the weights

Okay will add the conversion script too!!

IMvision12 avatar Dec 28 '22 20:12 IMvision12

@IMvision12 Have you checked ? https://github.com/sayakpaul/maxim-tf

innat avatar Dec 29 '22 08:12 innat

@IMvision12 Have you checked ? https://github.com/sayakpaul/maxim-tf

Oh we have tf implementation for maxim, because when I saw on paper with code there was only official flax implementation will refer that too thx for the info @innat. Also as we already have weights ported to TF and uploaded by @sayakpaul on TF-Hub do we still require the conversion script? @tanzhenyu

IMvision12 avatar Dec 29 '22 10:12 IMvision12