1.1.1, 1.1.0 errors on every model. But no error in 0.9.9

[Phantamoss]

With the 1.1.1 update, I get the same error on every model, while training on existing models.

Invalid characters in text (for alphabet): ’ (RIGHT SINGLE QUOTATION MARK),” (RIGHT DOUBLE QUOTATION MARK),“ (LEFT DOUBLE QUOTATION MARK),– (EN DASH),ñ (LATIN SMALL LETTER N WITH TILDE),é (LATIN SMALL LETTER E WITH ACUTE),‘ (LEFT SINGLE QUOTATION MARK),— (EM DASH),ï (LATIN SMALL LETTER I WITH DIAERESIS),… (HORIZONTAL ELLIPSIS)

1.1.0 generates the following error on every existing model.

Invalid characters in text (for alphabet): – (EN DASH),è (LATIN SMALL LETTER E WITH GRAVE),ô (LATIN SMALL LETTER O WITH CIRCUMFLEX),ç (LATIN SMALL LETTER C WITH CEDILLA),ñ (LATIN SMALL LETTER N WITH TILDE),é (LATIN SMALL LETTER E WITH ACUTE),… (HORIZONTAL ELLIPSIS),‘ (LEFT SINGLE QUOTATION MARK),” (RIGHT DOUBLE QUOTATION MARK),“ (LEFT DOUBLE QUOTATION MARK),’ (RIGHT SINGLE QUOTATION MARK),ï (LATIN SMALL LETTER I WITH DIAERESIS),à (LATIN SMALL LETTER A WITH GRAVE),— (EM DASH),î (LATIN SMALL LETTER I WITH CIRCUMFLEX)

0.9.9, 1.02 and 1.0.0 trains with no issue. At first I thought maybe it was created because of the removal of the models in 1.0.2 but it works in 1.0.2

[Phantamoss]

1.0.3 generates the following error.

Invalid characters (for alphabet): ’,‘,—,“,”, ,é,–,…

[Phantamoss]

I have gone through all of the training text, and looked for any of the characters that the errors list. And I cannot find any.

[BenAAndrew]

Hi @Phantamoss,

What this means is that in the dataset you have there are some invalid characters in the text. Open your dataset metadata.csv in notepad and see if you can ctrl+f for some of these characters (i.e. é) and remove them