Gmail settings not visible via /admin/mailsettings only SMTP set-up

[jasehigh]

Due to Gmail removing Simple Mail Transfer Protocol (SMTP) support I have upgraded to version 0.6.18 of calibre-web to set-up Gmail using the instructions available here: https://github.com/janeczku/calibre-web/wiki/Setup-Mailserver

I'm running on a Mac Mini (macOS 10.10.5) and have also upgraded to python 3 to run the latest version of calibre-web.

I have set-up the app in the Google Developer Console as described and believe everything is setup correctly:

I've installed installed google-auth-oauthlib and google-api-python-client from the optional-requirements.txt and have downloaded the JSON file (renamed gmail.json) and copied this to my ".calibre-web" folder.

However, when I restart calibre-web and go to /admin/mailsettings I don't the "Server Type" to set it to "Gmail Account with OAuth2 Verification". No option appears to click on "Setup Gmail Account".

Not sure what step I missed, but any assistance is appreciated.

Jason.

[OzzieIsaacs]

Please switch the log level to debug. Upon restart there should be a message get logged like the following: DEBUG {cps.services:49} Cannot import gmail, sending books via Gmail Oauth2 Verification will not work: No module named 'google_auth_oauthlib' This should help to get one step ahead

[jasehigh]

Yep, the log file does indeed show:

Cannot import gmail, sending books via Gmail Oauth2 Verification will not work: No module named 'google_auth_oauthlib'

Jason.

[2022-05-03 19:53:29,024]  INFO {cps:141} Starting Calibre Web...
[2022-05-03 19:53:29,365]  INFO {cps.server:228} Starting Tornado server on :8081
[2022-05-03 19:53:33,370] DEBUG {cps.config_sql:337} _ConfigSQL updating storage
[2022-05-03 19:53:35,776] DEBUG {cps.updater:324} Stable version: {'version': '0.6.18'}
[2022-05-03 19:53:42,007]  INFO {cps.server:274} webserver stop (restart=True)
[2022-05-03 19:53:42,011]  INFO {cps.server:262} Performing restart of Calibre-Web
[2022-05-03 19:53:43,599] DEBUG {cps.services:27} Cannot import goodreads, showing authors-metadata will not work: No module named 'betterreads'
[2022-05-03 19:53:43,600] DEBUG {cps.services:35} Cannot import simpleldap, logging in with ldap will not work: No module named 'flask_simpleldap'
[2022-05-03 19:53:43,601] DEBUG {cps.services:43} Cannot import SyncToken, syncing books with Kobo Devices will not work: No module named 'jsonschema'
**[2022-05-03 19:53:43,601] DEBUG {cps.services:50} Cannot import gmail, sending books via Gmail Oauth2 Verification will not work: No module named 'google_auth_oauthlib'**
[2022-05-03 19:53:43,813] DEBUG {cps.gdriveutils:84} Cannot import pydrive, httplib2, using gdrive will not work: No module named 'pydrive'
[2022-05-03 19:53:43,918] DEBUG {cps.helper:70} Cannot import Image, generating covers from non jpg files will not work: MagickWand shared library not found.
You probably had not installed ImageMagick library.
Try to install:
  https://docs.wand-py.org/en/latest/guide/install.html#install-imagemagick-on-mac
[2022-05-03 19:53:44,055] DEBUG {cps.comic:40} Cannot import comicapi, extracting comic metadata will not work: No module named 'comicapi'
[2022-05-03 19:53:44,058] DEBUG {cps.comic:47} Cannot import rarfile, extracting cover files from rar files will not work: No module named 'rarfile'
[2022-05-03 19:53:44,111] DEBUG {cps.uploader:42} Cannot import Image, generating pdf covers for pdf uploads will not work: MagickWand shared library not found.
You probably had not installed ImageMagick library.
Try to install:
  https://docs.wand-py.org/en/latest/guide/install.html#install-imagemagick-on-mac
[2022-05-03 19:53:44,195] DEBUG {cps.admin:72} Cannot import Flask-Dance, login with Oauth will not work: No module named 'flask_dance'
[2022-05-03 19:53:44,204] DEBUG {cps.gdrive:44} Cannot import googleapiclient, using GDrive will not work: No module named 'googleapiclient'
[2022-05-03 19:53:44,260] ERROR {cps.search_metadata:61} Import error for metadata source: amazon - No module named 'bs4'
[2022-05-03 19:53:44,264] ERROR {cps.search_metadata:61} Import error for metadata source: lubimyczytac - No module named 'dateutil'
[2022-05-03 19:53:44,265] ERROR {cps.search_metadata:61} Import error for metadata source: scholar - No module named 'scholarly'
[2022-05-03 19:53:44,402]  INFO {cps:136} *** "Flask-Login" version does not fit the requirements. Should: <0.5.1, Found: 0.6.0, please consider installing required version ***
[2022-05-03 19:53:44,402]  INFO {cps:136} *** "Flask" version does not fit the requirements. Should: <2.1.0, Found: 2.1.2, please consider installing required version ***
[2022-05-03 19:53:44,402]  INFO {cps:141} Starting Calibre Web...
[2022-05-03 19:53:44,725]  INFO {cps.server:228} Starting Tornado server on :8081
[2022-05-03 19:54:16,107] DEBUG {cps.updater:324} Stable version: {'version': '0.6.18'}

[OzzieIsaacs]

In this case something went wrong while installing google_auth_oauthlib. Are you using the system python an installed it in a python virtual environment, or vice versa?

[jasehigh]

Needed to install:

pip3 install google-auth-oauthlib
pip3 install google-api-python-client
pip3 install Flask-Dance

Can now see the Server Type option under Email-Server Settings. All working as expected.

[jasehigh]

So, sending books to my Kindle via gmail has stopped working.

When I send a book, I just get a status of "Failed" in the Tasks list.

Logs are telling me the following modules are missing, not sure if this is an issue:

[2022-06-05 17:44:10,219] ERROR {cps.search_metadata:61} Import error for metadata source: amazon - No module named 'bs4'
[2022-06-05 17:44:10,280] ERROR {cps.search_metadata:61} Import error for metadata source: lubimyczytac - No module named 'dateutil'
[2022-06-05 17:44:10,303] ERROR {cps.search_metadata:61} Import error for metadata source: scholar - No module named 'scholarly'

Also, tells me I have a higher version of some elements than required, but this was the case before, and no issues were present:

[2022-06-05 17:44:10,379]  INFO {cps:136} *** "Flask-Login" version does not fit the requirements. Should: <0.5.1, Found: 0.6.0, please consider installing required version ***
[2022-06-05 17:44:10,379]  INFO {cps:136} *** "Flask" version does not fit the requirements. Should: <2.1.0, Found: 2.1.2, please consider installing required version ***
[2022-06-05 17:44:10,380]  INFO {cps:136} *** "google-api-python-client" version does not fit the requirements. Should: <2.43.0, Found: 2.47.0, please consider installing required version ***
[2022-06-05 17:44:10,380]  INFO {cps:136} *** "google-api-python-client" version does not fit the requirements. Should: <2.43.0, Found: 2.47.0, please consider installing required version ***
[2022-06-05 17:44:10,380]  INFO {cps:136} *** "Flask-Dance" version does not fit the requirements. Should: <5.2.0, Found: 6.0.0, please consider installing required version ***
[2022-06-05 17:44:10,380]  INFO {cps:141} Starting Calibre Web...
[2022-06-05 17:44:10,838]  INFO {cps.server:228} Starting Tornado server on :8081

Nothing in the logs to indicate a failure when sending a book to my kindle:

[2022-06-05 17:48:29,071] DEBUG {cps.services.worker:88} Add Task for user: jasonhighfield - E-mail E-mail, Send to Kindle
[2022-06-05 17:48:29,157] DEBUG {cps.services.gmail:81} Start sending e-mail via Gmail
[2022-06-05 17:48:44,213] DEBUG {cps.updater:324} Stable version: {'version': '0.6.18'}

Checked my Google Cloud Developers console and everything appears to be set-up as per these instructions (https://github.com/janeczku/calibre-web/wiki/Setup-Mailserver) not logged in here this I set this up and confirmed everything was working.

At a loss as to why this is now failing. Any suggestions are very welcome.

[OzzieIsaacs]

First block search data metadata: no issue, only relevant for pulling metadata from the web

Second block: I pinned the max version to prevent issues with incompatible changes of future version. All versions you are having are „released“ in the nightly versions, so also no issue.

Amazon is changing the allowed format for receiving emails on your kindle to epub. If you update to the newest nightly version this is enabled and worth a try.

[jasehigh]

Updated to version "6/5/22, 9:06 AM" from the nightly build, and selected the "EPUB" option when sending to kindle.... still get the status "Failed" in the task list. Although the status is "Failed" the Run Time continues to count up, prevent any further attempts until I restart. Tried a different book and both EPUB and MOBI formats just in case)... same issue.

[OzzieIsaacs]

The infinite email send task should be saved