mailctl does not seem to work after some time?

[balaji-dutt]

Hi,

Apologies in advance for the poor issue title but I'm really not sure what is the actual problem and how to debug further. I'm currently using mailctl version 0.9.2 on Debian 12 with offlineimap. Here is how I have setup config.yaml and services.yaml

config.yaml:

# The primary purpose of mailctl is providing IMAP/SMTP clients with the
# capabilities of renewal and authorization of OAuth2 credentials.
# Accordingly only the OAuth2 related config entries are mandatory.

# Any path can be absolute or rooted to the home directory
# using tilde expansion (~/...)

services_file: ~/.config/mailctl/services.yaml

# Saving OAuth2 credentials in files encrypted by GPG

oauth2_dir: ~/.local/var/mailctl
encrypt_cmd:
  exec: gpg
  args:
    - --pinentry-mode=loopback
    - --passphrase-file=/path/to/gpgpass
    - --encrypt
    - --recipient
    - {GPGID}
    - -o
decrypt_cmd:
  exec: gpg
  args:
    - --pinentry-mode=loopback
    - --passphrase-file=/path/to/gpgpass
    - --decrypt

services.yaml:

gmail-backup:
  auth_endpoint: https://accounts.google.com/o/oauth2/auth
  auth_http_method: POST
  auth_params_mode: query-string
  token_endpoint: https://accounts.google.com/o/oauth2/token
  token_http_method: POST
  token_params_mode: both
  redirect_uri: http://localhost:8080
  auth_scope: https://mail.google.com/
  client_id: XXXXXXXXXXXX-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.apps.googleusercontent.com
  client_secret: GXXXXX-XXXXXXXXXXXXXXXXXXXXXXXXXXXX

I then added the following to .offlineimaprc:

[general]
accounts = gmail-backup
maxsyncaccounts = 1

## NB: only TTYUI and Blinkenlights are capable of prompting for the password
ui = ttyui

# use mailctl with offlineimap
pythonfile = ~/bin/offlineimaptoken.py

The contents of offlineimaptoken.py are as follows:

import subprocess

def get_token(email_address):
    return subprocess.run(["mailctl", "access", email_address], capture_output=True, text=True).stdout

The initial mailctl authorize [email protected] flow worked fine and it seemed like offlineimap was able to access Gmail as well. However, after several hours I noticed that offlineimap was erroring out with the following message:

  File "/usr/share/offlineimap3/offlineimap/imapserver.py", line 593, in acquireconnection
    self.__authn_helper(imapobj)
  File "/usr/share/offlineimap3/offlineimap/imapserver.py", line 453, in __authn_helper
    raise OfflineImapError("All authentication types "
offlineimap.error.OfflineImapError: All authentication types failed:
        XOAUTH2: b'[AUTHENTICATIONFAILED] Invalid credentials (Failure)'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/share/offlineimap3/offlineimap/accounts.py", line 658, in syncfolder
    remotefolder.syncmessagesto(localfolder, statusfolder)
  File "/usr/share/offlineimap3/offlineimap/folder/Base.py", line 1164, in syncmessagesto
    action(dstfolder, statusfolder)
  File "/usr/share/offlineimap3/offlineimap/folder/Base.py", line 991, in __syncmessagesto_copy
    self.copymessageto(uid, dstfolder, statusfolder, register=0)
  File "/usr/share/offlineimap3/offlineimap/folder/Gmail.py", line 293, in copymessageto
    super(GmailFolder, self).copymessageto(uid, dstfolder, statusfolder, register)
  File "/usr/share/offlineimap3/offlineimap/folder/Base.py", line 810, in copymessageto
    message = self.getmessage(uid)
              ^^^^^^^^^^^^^^^^^^^^
  File "/usr/share/offlineimap3/offlineimap/folder/Gmail.py", line 69, in getmessage
    data = self._fetch_from_imap(str(uid), self.retrycount)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/share/offlineimap3/offlineimap/folder/IMAP.py", line 869, in _fetch_from_imap
    self.imapserver.releaseconnection(imapobj)
  File "/usr/share/offlineimap3/offlineimap/imapserver.py", line 775, in releaseconnection
    self.assignedconnections.remove(connection)
ValueError: list.remove(x): x not in list

  list.remove(x): x not in list

I then tried to run mailctl access [email protected] but it gave the following error:

mailctl: getEmailAuth:
renewAccessToken: Nothing as refresh token argument
CallStack (from HasCallStack):
  error, called at lib/MailCtl/Authorization.hs:136:22 in mailctl-0.9.2-inplace:MailCtl.Authorization

I'm not clear what I'm missing here so would appreciate some tips on how to proceed further.

[pdobsan]

In your service.yaml file the root key must refer to the service (google in your case) and not to your email account (gmail-backup). Fix that and rerun the authorize process.

[balaji-dutt]

Hi @pdobsan I had changed the root key to gmail-backup as I have multiple Gmail accounts I'm trying to access with mailctl. I assumed therefore I cannot use the google root key. Can you let me know if it possible to have multiple Google accounts defined in services.yaml and if so how I should distinguish them?

[rmunn]

I believe this is happening because mailctl isn't receiving a refresh token from Google, and therefore it can't extend the life of the access token Google sends (which has a lifetime of 3600 seconds, i.e. 1 hour). I've opened a new issue, #36, with some suggestions about why this might be happening and what to do about it.

[pdobsan]

On Sat, Jan 27, 2024 at 06:57:54PM -0800, Balaji Dutt wrote:

Hi @pdobsan I had changed the root key to gmail-backup as I have multiple Gmail accounts I'm trying to access with mailctl. I assumed therefore I cannot use the google root key.

You assumed wrong.

The services.yaml describes your service provider's API, in your case it is google. You need google and only this one entry in your service.yaml file. You can have multiple account with google, they all use google as service provider.

Your accounts' data are not in the config files.