Peter

Hi, > Is there a julia version dependency? Looks like I forgot to add a julia version compat in Project.toml, but I'm also working on Julia v1.5. It should work...

> The problem is due to https://github.com/chengchingwen/Transformers.jl/blob/master/Project.toml#L33-L37 bounding CUDA packages and Adapt < 2 not allowing any recent Flux version to be used with this package. The gsoc2020 branch is...

Hi @bhalonen, Could you try the latest version to see if anything works fine?

> ... getting this package running smoothly ... Any issues?

Out of curiosity, you want to pre-train your own Bert model?

Should mostly look like this one https://github.com/chengchingwen/Transformers.jl/blob/master/example/BERT/cola/train.jl

Or you want to do token level classification?

> If you mean classifying parts of sentences as tokens, for example finding nouns, verbs, etc., Yes, that's what I means. > My current project is classifying sentences into a...

@jackn11 Holy! That is a bug for downloading the datasets. I was using the local files all the time so didn't notice it. For a quick workaround: 1. Download the...

The main issue for the pretrain example is that the code is really really old and I haven't find time to update that. But I didn't make it a high...