snorkel icon indicating copy to clipboard operation
snorkel copied to clipboard

Unicode characters break search

Open niteria opened this issue 4 years ago • 4 comments

Please see the attached screenshots.

The string I'm searching is "uńicode".

The error message is: "'ascii' codec can't encode character u'\u0144' in position 1: ordinal not in range(128)".

search-uńicode search-uńicode2

Logs from console:

ERROR INVOKING QuerySidebar run_query 'ascii' codec can't encode character u'\u0144' in position 1: ordinal not in range(128)                                                                                     
Traceback (most recent call last):                                                                                                                                                                                
  File "/usr/local/lib/python2.7/dist-packages/pudgy/blueprint.py", line 118, in invoke                                                                                                                           
    ret, proxy = found.invoke(cid, fn, args, kwargs)                                                                                                                                                              
  File "/usr/local/lib/python2.7/dist-packages/pudgy/components/bridge.py", line 119, in invoke                                                                                                                   
    return cls.__remote_calls__[fn](*args, **kwargs), c                                                                                                                                                           
  File "/usr/local/lib/python2.7/dist-packages/snorkel/auth.py", line 113, in wrapped_func                                                                                                                        
    r = f(*args, **kwargs)                                                                                                                                                                                        
  File "/usr/local/lib/python2.7/dist-packages/snorkel/auth.py", line 102, in wrapped_func                                                                                                                        
    return func(*args, **kwargs)                                                                                                                                                                                  
  File "/usr/local/lib/python2.7/dist-packages/snorkel/pages.py", line 193, in run_query                                                                                                                          
    query.add('filters', string_dict(filters))                                                                                                                                                                    
  File "/usr/local/lib/python2.7/dist-packages/snorkel/util.py", line 48, in convert                                                                                                                              
    return dict(map(convert, data.items()))                                                                                                                                                                       
  File "/usr/local/lib/python2.7/dist-packages/snorkel/util.py", line 50, in convert                                                                                                                              
    return type(data)(map(convert, data))                                                                                                                                                                         
  File "/usr/local/lib/python2.7/dist-packages/snorkel/util.py", line 50, in convert                                                                                                                              
    return type(data)(map(convert, data))                                                                                                                                                                         
  File "/usr/local/lib/python2.7/dist-packages/snorkel/util.py", line 50, in convert                                                                                                                              
    return type(data)(map(convert, data))                                                                                                                                                                         
  File "/usr/local/lib/python2.7/dist-packages/snorkel/util.py", line 46, in convert                                                                                                                              
    return str(data)                                                                                                                                                                                              
UnicodeEncodeError: 'ascii' codec can't encode character u'\u0144' in position 1: ordinal not in range(128)       

niteria avatar Apr 27 '20 13:04 niteria

If I build/run the python3 version with make virtualenv3 && make binary3 && sudo pip3 install dist/current/snorkel_lite-current-py3-none-any.whl && /usr/local/bin/snorkel.frontend version, it works.

niteria avatar Apr 27 '20 13:04 niteria

this is painful to fix in python2, but am looking into it

okayzed avatar May 20 '20 14:05 okayzed

Do you want to support python2? A lot of projects don't: https://python3statement.org/

niteria avatar May 20 '20 17:05 niteria

personally, i want to support python2, but i understand that everyone is moving to python3. it does make stuff difficult to support both versions, though.

okayzed avatar May 25 '20 23:05 okayzed