Genomic-ULMFiT icon indicating copy to clipboard operation
Genomic-ULMFiT copied to clipboard

Published 20 hours ago • verified publisher icon kheyer

State of the art?

Open lucidrains opened this issue 4 years ago • 2 comments

Hi! Thanks for your great work. I was wondering if this pre-trained network is state of the art or close? I am interested in doing some pre-training with biological sequences, and could use some guidance. Thank you!

lucidrains avatar Mar 06 '20 18:03 lucidrains

Hi @lucidrains Genomic-UMLFiT is a good guide for pre-training with genomic sequences. If you are interested in guidance for pre-training protein sequences, I would recommend Unified rational protein engineering with sequence-based deep representation learning. You can download their pre-trained models here.

djinnome avatar Mar 06 '20 20:03 djinnome

@djinnome Thanks for the response! I'm actually interested in both, and I'm aware of both UniRep and Berkeley's TAPE for protein sequences

lucidrains avatar Mar 06 '20 22:03 lucidrains