Transformers.jl icon indicating copy to clipboard operation
Transformers.jl copied to clipboard

Transformer tutorial is outdated

Open jackn11 opened this issue 2 years ago • 2 comments

I get a ERROR: syntax: invalid syntax "preprocess.(sample_data()) = ..." on the last line when I run the code below. The code is from the tutorial on this page https://chengchingwen.github.io/Transformers.jl/dev/tutorial/#Transformers.jl.

using Transformers
using Transformers.Basic
using Transformers.Pretrain
using CUDA
using Flux
enable_gpu(true)

labels = map(string, 1:10)
startsym = "11"
endsym = "12"
unksym = "0"
labels = [unksym, startsym, endsym, labels...]
vocab = Vocabulary(labels, unksym)

#function for generate training datas
sample_data() = (d = map(string, rand(1:10, 10)); (d,d))
#function for adding start & end symbol
preprocess(x) = [startsym, x..., endsym]

@show sample = preprocess.(sample_data())
@show encoded_sample = vocab(sample[1]) #use Vocabulary to encode the training data

sample = preprocess.(sample_data()) = (["11", "10", "8", "1", "10", "7", "10", "4", "2", "3", "3", "12"], ["11", "10", "8", "1", "10", "7", "10", "4", "2", "3", "3", "12"])

jackn11 avatar Jul 04 '22 15:07 jackn11

The code in tutorial is outdated, please refer to the example folder for latest workable code https://github.com/chengchingwen/Transformers.jl/tree/master/example/AttentionIsAllYouNeed

chengchingwen avatar Jul 04 '22 16:07 chengchingwen

Thank you for letting me know. It would definitely be useful to have a working tutorial on the docs and until then remove the outdated documentation or at least have some sort of notice.

jackn11 avatar Jul 04 '22 16:07 jackn11