Python 3 Support

[pganssle]

This app does not seem to work with Python 3. I think it would be a good idea to have at least basic polyglot Python 2/3 support.

I also think running using Python 3 or at least emulating python 3 would make things a bit less thorny when dealing with, for example, unicode filenames.

[pganssle]

I guess actually running this under Python 3 will be a problem since it pulls in anki as a dependency, and Anki still does not support Python 3. Probably not a bad idea to still use polyglot code in preparation for the eventuality that Anki is ported to Python 3.

I gave it a quick look and it seems like most places just need print x -> print(x) and except Exception, e -> except Exception as e.

[dsnopek]

So long as this will still work with the default Python 2 installed on the most common Linux distributions, this sounds fine. PRs welcome! :-)

[pganssle]

Yeah, I don't see much in the way of Python 3 exclusive stuff out there, so there's a good number of idioms and libraries (e.g. six) that help you bridge the gap for writing polyglot 2/3 code.

It kinda surprises me any time I see print x, honestly, because for most uses of print you can just add parens and it makes it 2/3 compatible even without importing print_function from __future__ (though there are a few places in this application that would require that, since you print to a stream).

I can work up a quick PR, but it's tough to know how thorough a job I'd be doing without being able to actually run it with Python 3.

[cdpm]

Anki Desktop is being ported to Python 3: https://github.com/dae/anki/commit/15b349e3a8b34bf80c134b406c9b90f61250ee9e.

[tsudoko]

I have started porting anki-sync-server to Python 3. It currently works, but hasn't been tested very thoroughly. If there's anyone interested in improving the code, it's at https://github.com/tsudoko/anki-sync-server/tree/anki_2_1. I'll make a PR later if the code is considered good enough.

[colonelsmoothie]

Hello,

Thanks for your work, @tsudoko. I've been testing out your fork, but I'm having trouble starting the sever. Here's what I did:

I installed both the latest Anki Alpha version and tsudoko's Python 3 fork:

git clone https://github.com/dae/anki
cd anki
sudo make install

cd ~
git clone https://github.com/tsudoko/anki-sync-server
sudo pip install git+file:///home/test/anki-sync-server

Then I created the production.ini file per the instructions. But when I tried to run the server under debug:

cd /usr/local/bin
sudo python3 ./ankiserverctl.py adduser test
sudo python3 ./ankiserverctl.py debug

I get the following error:

/usr/local/bin$ sudo python3 ./ankiserverctl.py debug
Traceback (most recent call last):
  File "/usr/local/bin/paster", line 11, in <module>
    sys.exit(run())
  File "/usr/local/lib/python2.7/dist-packages/paste/script/command.py", line 102, in run
    invoke(command, command_name, options, args[1:])
  File "/usr/local/lib/python2.7/dist-packages/paste/script/command.py", line 141, in invoke
    exit_code = runner.run(args)
  File "/usr/local/lib/python2.7/dist-packages/paste/script/command.py", line 236, in run
    result = self.command()
  File "/usr/local/lib/python2.7/dist-packages/paste/script/serve.py", line 284, in command
    relative_to=base, global_conf=vars)
  File "/usr/local/lib/python2.7/dist-packages/paste/script/serve.py", line 329, in loadapp
    **kw)
  File "/usr/local/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 247, in loadapp
    return loadobj(APP, uri, name=name, **kw)
  File "/usr/local/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 272, in loadobj
    return context.create()
  File "/usr/local/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 710, in create
    return self.object_type.invoke(self)
  File "/usr/local/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 217, in invoke
    next_app = context.next_context.create()
  File "/usr/local/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 710, in create
    return self.object_type.invoke(self)
  File "/usr/local/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 144, in invoke
    **context.local_conf)
  File "/usr/local/lib/python2.7/dist-packages/paste/deploy/util.py", line 55, in fix_call
    val = callable(*args, **kw)
  File "/usr/local/lib/python2.7/dist-packages/paste/urlmap.py", line 31, in urlmap_factory
    app = loader.get_app(app_name, global_conf=global_conf)
  File "/usr/local/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 350, in get_app
    name=name, global_conf=global_conf).create()
  File "/usr/local/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 362, in app_context
    APP, name=name, global_conf=global_conf)
  File "/usr/local/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 454, in get_context
    section)
  File "/usr/local/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 476, in _context_from_use
    object_type, name=use, global_conf=global_conf)
  File "/usr/local/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 406, in get_context
    global_conf=global_conf)
  File "/usr/local/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 296, in loadcontext
    global_conf=global_conf)
  File "/usr/local/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 328, in _loadegg
    return loader.get_context(object_type, name, global_conf)
  File "/usr/local/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 620, in get_context
    object_type, name=name)
  File "/usr/local/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 646, in find_egg_entry_point
    possible.append((entry.load(), protocol, entry.name))
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2229, in load
    return self.resolve()
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2235, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/local/lib/python2.7/dist-packages/AnkiServer/apps/sync_app.py", line 36, in <module>
    import anki
  File "/usr/share/anki/anki/__init__.py", line 8, in <module>
    raise Exception("Anki should be run with Python 3")
Exception: Anki should be run with Python 3

Thanks!

[tsudoko]

Are you sure you're using the right version of pip? I managed to reproduce the issue in a clean Debian stretch chroot by running pip for Python 2 instead of the Python 3 one. On most Linux distributions the Python 3 pip has a binary called pip3, you might need to install it before you can use it.

[colonelsmoothie]

Thanks for the quick reply! It looks like I should have used pip3. We're making progress, but I still have issues getting things running:

sudo apt-get install python3-pip
sudo pip3 install git+file:///home/test/anki-sync-server

Then I set up production.ini using the contents of example.ini:

cd /usr/local/bin
sudo touch production.ini
sudo gedit production.ini

Then I configured the server:

sudo ./ankiserverctl.py adduser test
sudo ./ankiserverctl.py debug

I ended up getting an error message saying I needed pyaudio, which I then installed

sudo apt-get install python3-pyaudio
sudo ./ankiserverctl.py debug

Which led to...

Starting server in PID 7029.
serving on http://127.0.0.1:27701

Woohoo! let's start the server:

sudo ./ankiserverctl.py start

I then downloaded the Anki 2.1 Alpha 8 here https://apps.ankiweb.net/downloads/alpha/alpha8/

Started Anki, then closed it.

cd ~/Downloads/anki-2.1.0a8-amd64/bin
./anki

Then I added mysyncserver.py to the ~/Documents/Anki/addons21/ folder with the following contents:

import anki.sync
anki.sync.SYNC_BASE = 'http://127.0.0.1:27701/'
anki.sync.SYNC_MEDIA_BASE = 'http://127.0.0.1:27701/msync/'

Started up anki

cd ~/Downloads/anki-2.1.0a8-amd64/bin
./anki

Tried to sync with server and got the following message

Syncing failed:
AnkiWeb encountered an error. Please try again in a few minutes, and if the problem persists, please file a bug report.

Then I tried a couple of things

sudo ufw allow 27701

and changing the contents of mysyncserver.py to:

import anki.sync
anki.sync.SYNC_BASE = 'http://10.0.2.15:27701/'
anki.sync.SYNC_MEDIA_BASE = 'http://10.0.2.15:27701/msync/'

When I tried to sync again, I got the following error

Syncing failed:
Traceback (most recent call last):
  File "site-packages/requests/packages/urllib3/connection.py", line 138, in _new_conn
  File "site-packages/requests/packages/urllib3/util/connection.py", line 98, in create_connection
  File "site-packages/requests/packages/urllib3/util/connection.py", line 88, in create_connection
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "site-packages/requests/packages/urllib3/connectionpool.py", line 594, in urlopen
  File "site-packages/requests/packages/urllib3/connectionpool.py", line 361, in _make_request
  File "http/client.py", line 1083, in request
  File "http/client.py", line 1128, in _send_request
  File "http/client.py", line 1079, in endheaders
  File "http/client.py", line 911, in _send_output
  File "http/client.py", line 854, in send
  File "site-packages/requests/packages/urllib3/connection.py", line 163, in connect
  File "site-packages/requests/packages/urllib3/connection.py", line 147, in _new_conn
requests.packages.urllib3.exceptions.NewConnectionError: <requests.packages.urllib3.connection.HTTPConnection object at 0x7fdae21e6b38>: Failed to establish a new connection: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "site-packages/requests/adapters.py", line 423, in send
  File "site-packages/requests/packages/urllib3/connectionpool.py", line 643, in urlopen
  File "site-packages/requests/packages/urllib3/util/retry.py", line 363, in increment
requests.packages.urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='10.0.2.15', port=27701): Max retries exceeded with url: /sync/hostKey (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7fdae21e6b38>: Failed to establish a new connection: [Errno 111] Connection refused',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "aqt/sync.py", line 323, in run
  File "aqt/sync.py", line 338, in _sync
  File "anki/sync.py", line 587, in hostKey
  File "anki/sync.py", line 561, in req
  File "anki/sync.py", line 468, in post
  File "site-packages/requests/sessions.py", line 535, in post
  File "site-packages/requests/sessions.py", line 488, in request
  File "site-packages/requests/sessions.py", line 609, in send
  File "site-packages/requests/adapters.py", line 487, in send
requests.exceptions.ConnectionError: HTTPConnectionPool(host='10.0.2.15', port=27701): Max retries exceeded with url: /sync/hostKey (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7fdae21e6b38>: Failed to establish a new connection: [Errno 111] Connection refused',))

[colonelsmoothie]

Some success - I have been able to get @tsudoko's version of the server to sync with Anki 2.0.39. However, I still haven't gotten it to sync with Anki 2.1.0a8.

I did have to change the dependency WebOb in setup.py to an older version 1.6.0. There was a new version 1.7.0 released after this fork was created that causes an error.

I changed

    install_requires=[
        "PasteDeploy>=1.3.2",
        "PasteScript>=1.7.3",
        "WebOb>=0.9.7",
        "SQLAlchemy>=0.6.3",
],

to

    install_requires=[
        "PasteDeploy>=1.3.2",
        "PasteScript>=1.7.3",
        "WebOb==1.6.0",
        "SQLAlchemy>=0.6.3",
],

That gets Anki 2.0.39 to sync with the server. Using the latest WebOb 1.7.0 causes the following error when I try to run the server in debug mode:

Starting server in PID 11993.
serving on http://10.0.2.15:27701
----------------------------------------
Exception happened during processing of request from ('10.0.2.15', 60504)
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/paste/httpserver.py", line 304, in wsgi_execute
    self.wsgi_start_response)
  File "/usr/local/lib/python3.5/dist-packages/paste/translogger.py", line 69, in __call__
    return self.application(environ, replacement_start_response)
  File "/usr/local/lib/python3.5/dist-packages/paste/urlmap.py", line 216, in __call__
    return app(environ, start_response)
  File "/usr/local/lib/python3.5/dist-packages/webob/dec.py", line 131, in __call__
    resp = self.call_func(req, *args, **self.kwargs)
  File "/usr/local/lib/python3.5/dist-packages/webob/dec.py", line 196, in call_func
    return self.func(req, *args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/AnkiServer/apps/sync_app.py", line 544, in __call__
    body=json.dumps(result))
  File "/usr/local/lib/python3.5/dist-packages/webob/response.py", line 310, in __init__
    "You cannot set the body to a text value without a "
TypeError: You cannot set the body to a text value without a charset

Even after changing WebOb to 1.6.0, I still can't get Anki 2.1.0a8 to sync. The program displays the message:

Syncing failed: 'list indices must be integers or slices, not str'

Unfortunately I do not know how to display the actual error that occurred.

The server itself does not display any error messages in debug mode.

[tsudoko]

Thank you for actually trying it out. Both problems should be fixed now. By the way, it might be better to report further issues at tsudoko/anki-sync-server if they're related to my Python 3 fork, at least for now.

[colonelsmoothie]

Thanks for the fixes. I have gotten collections to sync back and forth with the latest revisions. I am getting some weird behavior from the GUI but maybe that's due to bugs in Anki alpha itself. I'll keep testing it and report issues in your fork if I find any.