Some unicode tags return empty slug

[tpeaton]

Certain unicode doesn't yield a usable slug.

>>> obj.tags.add('(ง’̀’́)ง')
>>> obj.tags.all()
[<Tag: ง’̀’́)ง>]
>>> obj.tags.all()[0].slug
u''

We've solved this internally by doing this:

from shortuuid import uuid

name = data['name'].encode('ascii', 'ignore').decode('utf-8')
if not taggit_slugify(name):
    data['slug'] = 'tag-{}'.format(uuid())

Seem reasonable? If so, I can add it to this method and create a PR.

[frewsxcv]

Do you have 'unidecode' installed?

[tpeaton]

I do not.

[frewsxcv]

So if you install that, it might fix this issue. We should probably add a note about it in the docs somewhere....

Relevant PR: https://github.com/alex/django-taggit/pull/315

[tpeaton]

That seems to work fine, thanks!

[uksmartsolutions]

Yes, installing "unidecode" fixed issues. (pip install unidecode)

[AliIslamov]

I have installed unidecode, but my ciryllic tags still have empty slugs.

[rtpg]

@AliIslamov could you provide a snippet of how you are creating your tags, along with an assertion that the slug is indeed the empty string?

In particular, could you try things like calling tag.slugify(tag_string) to confirm that the slugify method is returning an empty string?

[rtpg]

I would like to add more tests or something here to work through this issue but am having a hard time conceptualizing at what part of the system this is failing

[AliIslamov]

@AliIslamov could you provide a snippet of how you are creating your tags, along with an assertion that the slug is indeed the empty string?

In particular, could you try things like calling tag.slugify(tag_string) to confirm that the slugify method is returning an empty string?

I already have solved my problem.

1. Have written in settings.py:

TAGGIT_STRIP_UNICODE_WHEN_SLUGIFYING = True

2. Have installed unidecode:

pip install unidecode

3. Have written in models.py for my application:

from unidecode import unidecode

And now everything works well 👍

[AliIslamov]

I would like to add more tests or something here to work through this issue but am having a hard time conceptualizing at what part of the system this is failing

We can conclude that while unicode is enabled by default, cyrillic tags get an empty slug.

The situation is solved by forcing unicode to ascii convertation, installing the unidecode module and importing it into the models.py file, in which the TaggableManager() field is present.

In my opinion it makes sense to add this instruction in documentation.