Kobo Clara sync issue

[exile259]

Describe the bug/problem I am using the Linuxserver Calibre-web docker container with their SWAG nginx/letsencrypt container.I have followed the Nginx guide (https://github.com/janeczku/calibre-web/wiki/Setup-Reverse-Proxy), and configured my CW instance to allow sync for one bookshelf. Access logs show the sync attempt, but nothing ever syncs

To Reproduce Steps to reproduce the behavior:

1. unlock Kobo
2. Tap the sync button
3. Tap Sync Now
4. NO books are synced

Logfile

access logs: https://zerobin.net/?f7bc5221eb2269c0#02/aC1MdSjiVVlSsazmFjJtOmzSvhAqXT6o/fEB425c= Calibre logs: https://zerobin.net/?8e7415113c65b117#hX7AdBFj+irjs/Dw3S+lhvVKeO2rCFYPzm+j+IN4xQ0=

Here is my conf file for calibre. any deviation from the Nginx guide was to make the web portal function (following the instructions to remove the proxy_set_header and alter location resulted in no formatting being applied to the web portal)

## Version 2021/05/18
# calibre-web does not require a base url setting

location /calibre-web {
    return 301 $scheme://$host/calibre-web/;
}

location ^~ /calibre-web/ {
        client_max_body_size 20M;
                proxy_bind              $server_addr;
                proxy_pass http://###.###.###.###:8083;
                proxy_set_header        Host            $http_host;
                proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header        X-Scheme        $scheme;
                proxy_set_header        X-Script-Name   /calibre-web;  # IMPORTANT: path has NO trailing slash
    # enable the next two lines for http auth
    #auth_basic "Restricted";
    #auth_basic_user_file /config/nginx/.htpasswd;

    # enable the next two lines for ldap auth, also customize and enable ldap.conf in the default conf
    #auth_request /auth;
    #error_page 401 =200 /ldaplogin;

    # enable for Authelia, also enable authelia-server.conf in the default site config
    # To use Authelia to log in to Calibre-Web, make sure "Reverse Proxy Login" is 
    # enabled, "Reverse Proxy Header Name" is set to Remote-User, and each Authelia
    # user also has a corresponding user manually created in Calibre-Web.
    #include /config/nginx/authelia-location.conf;

    include /config/nginx/proxy.conf;
    include /config/nginx/resolver.conf;
    #set $upstream_app calibre-web;
    #set $upstream_port 8083;
    #set $upstream_proto http;
    #proxy_pass $upstream_proto://$upstream_app:$upstream_port;
#       proxy_pass http://192.168.1.220:8083;

#    proxy_set_header X-Scheme $scheme;
#    proxy_set_header X-Script-Name /calibre-web;
}

# OPDS feed for eBook reader apps
# Even if you use Authelia, the OPDS feed requires a password to be set for
# the user directly in Calibre-Web, as eBook reader apps don't support 

Expected behavior Calibre-web to sync expected shelf to my Kobo

Screenshots

Environment (please complete the following information):

• OS:Unraid 6.9.2
• Calibre-Web version: 0.6.18 :
• Docker container:lscr.io/linuxserver/calibre-web:
• Browser: Kobo Clara

Additional context Add any other context about the problem here. [e.g. access via reverse proxy, database background sync, special database location]

[OzzieIsaacs]

In both provided logfiles no request regarding kobo is logged, so all requests seem to be "lost between kobo and calibre-web".

Please checkout the procedure described here (https://github.com/janeczku/calibre-web/wiki/Kobo-Integration#checking-kobo-sync-content) to figure out if it's possible to access the kobo routines of calibre-web with a browser. The document is pretty new, so maybe there are bugs in it, so if it's not working out, please be patient and post the issue(s) here

[OzzieIsaacs]

Access logs show the sync attempt, but nothing ever syncs

Could you please tell me which line the sync shows, as I can't find it: ?

[simkin]

Fore me the initial transfer also failed and all future sync attempts resulted in nothing as calibre-web assumed the sync was already completed successfully. Please try to force a full sync

And experiment with your external port:

[Wummeke]

I am having the same problem too, I can't get sync working on my Clara HD, with a similar set up. I have a Docker container

version: '3'

services:
  calibre:
    container_name: calibre
    image: lscr.io/linuxserver/calibre-web:latest
    environment:
      - PUID=1026
      - PGID=100
      - TZ=Europe/Amsterdam
      - DOCKER_MODS=linuxserver/calibre-web:calibre #optional
      - OAUTHLIB_RELAX_TOKEN_SCOPE=1 #optional
    volumes:
      - /volume1/docker/calibre/config:/config
      - /volume1/Calibre Library:/books
      - /volume1/docker/calibre/config/gmail.json:/app/calibre-web/gmail.json
    ports:
      - 8083:8083
    networks:
      Network:
        ipv4_address: 172.19.0.9
    restart: unless-stopped
    
networks:
  Network:
    external: true
    name: iotstack_Network

which is accessible through a reverse proxy

## Version 2021/05/18
# calibre-web does not require a base url setting

location /calibre-web {
    return 301 $scheme://$host/calibre-web/;
}

location ^~ /calibre-web/ {
    # enable the next two lines for http auth
    #auth_basic "Restricted";
    #auth_basic_user_file /config/nginx/.htpasswd;

    # enable the next two lines for ldap auth, also customize and enable ldap.conf in the default conf
    #auth_request /auth;
    #error_page 401 =200 /ldaplogin;

    # enable for Authelia, also enable authelia-server.conf in the default site config
    # To use Authelia to log in to Calibre-Web, make sure "Reverse Proxy Login" is 
    # enabled, "Reverse Proxy Header Name" is set to Remote-User, and each Authelia
    # user also has a corresponding user manually created in Calibre-Web.
    #include /config/nginx/authelia-location.conf;

    include /config/nginx/proxy.conf;
    include /config/nginx/resolver.conf;
    set $upstream_app calibre;
    set $upstream_port 8083;
    set $upstream_proto http;
    proxy_pass $upstream_proto://$upstream_app:$upstream_port;

    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
	proxy_set_header X-Scheme $scheme;
    proxy_set_header X-Script-Name /calibre-web;
}

# OPDS feed for eBook reader apps
# Even if you use Authelia, the OPDS feed requires a password to be set for
# the user directly in Calibre-Web, as eBook reader apps don't support 
# form-based logins, only HTTP Basic auth.
location ^~ /calibre-web/opds/ {
    include /config/nginx/proxy.conf;
    include /config/nginx/resolver.conf;
    set $upstream_app calibre;
    set $upstream_port 8083;
    set $upstream_proto http;
    proxy_pass $upstream_proto://$upstream_app:$upstream_port;
    proxy_set_header X-Scheme $scheme;
    proxy_set_header X-Script-Name /calibre-web;
}

However, when I try to sync, the Clara HD says 'Sync failed, please try again' in my calibre-web.log I see the following:

[2022-06-17 21:48:42,989] DEBUG {cps.ub:82} Found stored session: 935bb429aa934863ca5d039336db1583f84b3514a2662163f43cc870aba2f9f02d781fc7357b4de5a1f636a6a5be13661d47fe5c15c459072fb932658f6114ef
[2022-06-17 21:48:42,989]  INFO {cps.kobo:1054} Init
[2022-06-17 21:48:43,224] DEBUG {cps.kobo:975} Unimplemented User Request received: https://www.mijn-url.nl/calibre-web/kobo/my_key/v1/user/profile
[2022-06-17 21:48:43,358] DEBUG {cps.kobo:1015} Unimplemented Products Request received: https://www.mijn-url.nl/calibre-web/kobo/my_key/v1/deals
[2022-06-17 21:48:43,542] DEBUG {cps.ub:82} Found stored session: 935bb429aa934863ca5d039336db1583f84b3514a2662163f43cc870aba2f9f02d781fc7357b4de5a1f636a6a5be13661d47fe5c15c459072fb932658f6114ef
[2022-06-17 21:48:43,543]  INFO {cps.kobo:140} Kobo library sync request received.
[2022-06-17 21:48:43,543] DEBUG {cps.kobo:141} SyncToken: 0001-01-01 00:00:00,0001-01-01 00:00:00,0001-01-01 00:00:00,0001-01-01 00:00:00,0001-01-01 00:00:00,[very_long_string_of_characters_here]
[2022-06-17 21:48:43,637] DEBUG {cps.kobo:215} Books to Sync: 3
[2022-06-17 21:48:43,859] DEBUG {cps.kobo:280} Remaining books to Sync: 0
[2022-06-17 21:49:10,932] DEBUG {cps.ub:82} Found stored session: 935bb429aa934863ca5d039336db1583f84b3514a2662163f43cc870aba2f9f02d781fc7357b4de5a1f636a6a5be13661d47fe5c15c459072fb932658f6114ef
[2022-06-17 21:49:10,933]  INFO {cps.kobo:1054} Init
[2022-06-17 21:49:10,972] DEBUG {cps.kobo:975} Unimplemented User Request received: https://www.mijn-url.nl/calibre-web/kobo/my_key/v1/user/profile
[2022-06-17 21:49:11,058] DEBUG {cps.kobo:1015} Unimplemented Products Request received: https://www.mijn-url.nl/calibre-web/kobo/my_key/v1/deals
[2022-06-17 21:49:11,240] DEBUG {cps.ub:82} Found stored session: 935bb429aa934863ca5d039336db1583f84b3514a2662163f43cc870aba2f9f02d781fc7357b4de5a1f636a6a5be13661d47fe5c15c459072fb932658f6114ef
[2022-06-17 21:49:11,240]  INFO {cps.kobo:140} Kobo library sync request received.
[2022-06-17 21:49:11,240] DEBUG {cps.kobo:141} SyncToken: 0001-01-01 00:00:00,0001-01-01 00:00:00,0001-01-01 00:00:00,0001-01-01 00:00:00,0001-01-01 00:00:00,[very_long_string_of_characters_here]
[2022-06-17 21:49:11,324] DEBUG {cps.kobo:215} Books to Sync: 0
[2022-06-17 21:49:11,385] DEBUG {cps.kobo:280} Remaining books to Sync: 0

Things I've tried:

Activated 'Force full kobo sync'; Created collection on my kobo with the same name as the shelve I'd like to sync, removed it later; Restarted the Clara HD; Restarted the Calibre-web container; Turned the option 'Proxy unknown requests to Kobo Store' off and on; Tried various ports for "Server External Port (for port forwarded API calls)" (ports 8083, 443, 80); Configured the Clara HD to sync locally, to bypass the reverse proxy; Turned off Adguard DNS filtering on my network.

All without luck so far. I haven't factory reset the Clara yet, because I have reading progress and books already on it, so I would rather not do that, as long as it is not necessary.

It turns out I just needed a rubber ducky to help me with debugging.

I had to add some proxy_buffer lines to my reverse proxy configuration, al stated in the documentation here:

proxy_buffer_size 128k;
proxy_buffers 4 256k;
proxy_busy_buffers_size 256k;

That solved my issue completely. I hope this could be useful for someone else too.