Compose functionality is not working on web and iOS

[mdecimus]

Description

The compose functionality does not seem to be working on both web and iOS clients. I tried on a Mac using Chrome, Firefox and Safari and then on an iPhone. This happens with the latest Docker image as of August 26th, 2022.

There are a number of issues:

• The Send button does nothing on both platforms. No errors are reported to the console.
• The Rich text editor is broken and does not allow any text input, it looks like a file is missing:

DevTools failed to load source map: Could not load content for https://jmap.example.org/assets/packages/html_editor_enhanced/assets/summernote-lite.min.js.map: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE

• The recipient lists expects either a space or a comma to accept a recipient. If I enter a valid email address and do not press space, I get an error indicating that I need to specify at least one recipient.

Reproduction Steps

Try sending an email from a Mac or iOS using the latest Docker image.

Context

• Mac (Chrome, Firefox and Safari).
• iPhone iOS

[hoangdat]

Hi @mdecimus, what image you are working on? master tag? First, for the stability, please use the docker with release or the highest version (Fx: v0.3.13). Second, we will take a look at the problems you reported. Thank you.

[mdecimus]

Hi @hoangdat,

Thank you for the quick reply.

I was indeed on the master tag (v.0.3.13) so I just switched to the release tag (also shows v.0.3.13) but unfortunately the problem persists. On iOS I am using the latest version available on the AppStore which is also v.0.3.13.

Just in case it helps you debug, when opening the Compose window the client is sending a JMAP request to fetch the identities. After that no further JMAP requests are sent and no errors are found on the console. When clicking on the Send button the following is dumped to the console:

ComposerController::_validateAttachmentsSize(): 0 main.dart.js:3233:71
ComposerController::_validateAttachmentsSize(): totalInlineAttachmentsSize: 0 main.dart.js:3233:71
ComposerController::_validateAttachmentsSize(): totalSizeReadyToUpload: 0 main.dart.js:3233:71
MailboxDashBoardController::maxSizeAttachmentsPerEmail(): UnsignedInt main.dart.js:3233:71

Thanks

[dab246]

Hi @mdecimus: Thanks for your feedback

• I tested it on my server and it works fine.
• If you have a development environment on your server, please provide me with account/password information so I can check it more closely.

[mdecimus]

Hi @dab246

Some background information, I am testing TMail against our new JMAP server that is going to be open sourced in the next few weeks. Our JMAP server passes all tests created by Fastmail and email submissions are working properly when using the Ltt.rs for Android client. Integration with Tmail-web works perfectly, only the compose functionality is giving problems.

Please feel free to use the following account to test TMail against our JMAP server, it contains the Enron dataset with 500k messages:

JMAP Address: https://jmap.cloud:8443/
Account: [email protected]
Pass: demo
TMail Web Address: https://jmap.cloud

Please note that even though email submissions are disabled on that server, EmailSubmission/set should still work but the message won't be delivered.

As a side note, our server supports JMAP over WebSocket, EventSource and Search snippets in case you want to add support for these functionalities to TMail in the future.

Please let me know if you need anything help, I'll be glad to help.

Thank you.

[dab246]

Hi @mdecimus

1. I'm trying to test it on the server on your account and noticed the following problems:

• In Default mailbox (all mailbox has role) do not have Outbox and Sent mailbox So it leads to email failed to send error

2. Our email delivery script is as follows:

• The email will be sent to the Outbox mailbox folder
• The email will then be moved from the Sent mailbox folder So we need Outbox and Sent mailbox folder to send email successfully

@chibenwa Do you have an opinion on this issue?

[mdecimus]

Hi @dab246 ,

I added a Drafts and Sent folders with the appropriate role set but the problem persists. I think I found the problem, TMail is sending the following invalid JMAP Email/set request:

{
    "using": [
        "urn:ietf:params:jmap:mail",
        "urn:ietf:params:jmap:core"
    ],
    "methodCalls": [
        [
            "Email/set",
            {
                "accountId": "d",
                "create": {
                    "8981aaa0-2829-11ed-989e-bb88753727a8": {
                        "id": "8981aaa0-2829-11ed-989e-bb88753727a8",
                        "mailboxIds": {
                            "dno": true
                        },
                        "keywords": {
                            "$draft": true
                        },
                        "subject": "fsdff",
                        "from": [
                            {
                                "name": null,
                                "email": "[email protected]"
                            }
                        ],
                        "to": [
                            {
                                "name": null,
                                "email": "[email protected]"
                            }
                        ],
                        "cc": [],
                        "bcc": [],
                        "replyTo": [
                            {
                                "name": null,
                                "email": "[email protected]"
                            }
                        ],
                        "htmlBody": [
                            {
                                "partId": "8981aaa1-2829-11ed-989e-bb88753727a8",
                                "blobId": "8981aaa2-2829-11ed-989e-bb88753727a8",
                                "type": "text/html"
                            }
                        ],
                        "bodyValues": {
                            "8981aaa1-2829-11ed-989e-bb88753727a8": {
                                "value": "<p>sdfdf<br></p>",
                                "isEncodingProblem": false,
                                "isTruncated": false
                            }
                        },
                        "header:User-Agent:asText": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:103.0) Gecko/20100101 Firefox/103.0"
                    }
                }
            },
            "c0"
        ]
    ]
}

Which has two problems:

                           "id": "8981aaa0-2829-11ed-989e-bb88753727a8",
...
                           {
                                "partId": "8981aaa1-2829-11ed-989e-bb88753727a8",
                                "blobId": "8981aaa2-2829-11ed-989e-bb88753727a8",
                                "type": "text/html"
                            }
...

• It is specifying an invalid id on the set request, which according to RFC8260 section 5.3 has to be rejected with an invalidProperties error: It contains a property that may only be set by the server (e.g., "id") and is different to the current value..
• It is including an htmlBody part with an invalid blobId and in addition to that also specifies a partId. According to RFC8621 The client may specify a partId OR a blobId, but not both..

One last comment regarding the missing folders. It would be nice if TMail could create any missing folders or report the error to the user instead of failing silently. Users may accidentally delete the Send and/or Draft folders and break the functionality.

Thank you.

[dab246]

Hi @mdecimus, Thank for your feedback. I will answer your questions as follows:

1. The inner object of the Create operator in the Email/set method must not have the value Id. So when you apply it to email you should remove it in your request. In our project we also don't pass id in request send email.
2. Exactly what you said. partId or blobId should only have one of the two in htmlBody. We will fix this in our project.
3. We are in the development phase so your suggestions will be applied by us in the next development. You can follow issues #901

• In addition, currently in our application, users are not allowed to delete or edit Default Mailbox such as (Inbox, Outbox, Sent, Draft). They can create, edit and delete mailbox folders that do not have Role. So when you use our app you don't need to worry about that.

• One more note, I see you create Draft and Sent mailbox but you still get that error. As I said earlier, we need the Outbox mailbox folder and the Sent mailbox folder which is the important mailbox for successful email delivery. Please create Outbox mailbox folder and do it again.

You can see our full email Request code below:

{
    "using": [
        "urn:ietf:params:jmap:core",
        "urn:ietf:params:jmap:mail",
        "urn:ietf:params:jmap:submission"
    ],
    "methodCalls": [
        [
            "Email/set",
            {
                "accountId": "{{accountId}}",
                "create": {
                    "aaaaaa": {
                        "mailboxIds": {
                            "5dfb3290-0a14-11ec-b57c-2fef1ee78d9e": true
                        },
                        "from":[{
                            "email":"[email protected]"
                        }],
                        "to":[{
                            "email":"[email protected]"
                        }],
                        "Subject":"test onSuccessUpdateEmail",
                        "htmlBody": [
                            {
                                "partId": "mmm",
                                "type": "text/html"
                            }
                        ],
                        "bodyValues": {
                            "mmm": {
                                "value": "<!DOCTYPE html> <html> <body> <p><b>body 1</b></p><br><br></body> </html>",
                                "isTruncated": false,
                                "isEncodingProblem": false
                            }
                        }
                    }
                }
            },
            "c1"
        ],
        [
            "EmailSubmission/set",
            {
                "accountId": "{{accountId}}",
                "create": {
                    "a1234": {
                        "emailId": "#aaaaaa",
                        "envelope": {
                            "mailFrom": {
                                "email": "[email protected]"
                            },
                            "rcptTo": [{
                                "email": "[email protected]"
                            }]
						}
                    }
                },
                "onSuccessUpdateEmail": {
                    "#a1234": {
                        "mailboxIds": {
                            "abb99c10-18d9-11eb-a677-2990b970028d": true
                        }
                    }
                }
            },
            "c2"
        ]
    ]
}

Thanks you.

[mdecimus]

Hi @dab246 ,

The inner object of the Create operator in the Email/set method must not have the value Id. So when you apply it to email you should remove it in your request. In our project we also don't pass id in request send email.

Let me clarify, when TMail sends the Email/set request to the JMAP server it is including an invalid id (in the example I included above it was 8981aaa0-2829-11ed-989e-bb88753727a8 which was not generated by the JMAP server). In order to make TMail compliant with RFC8620 an id should not be included in the set item request.

As I said earlier, we need the Outbox mailbox folder and the Sent mailbox folder which is the important mailbox for successful email delivery. Please create Outbox mailbox folder and do it again.

I could create a folder named "Outbox" but not a folder with the role "outbox" as it is not a valid role. RFC8621 section 2 specifies what kind of values can be passed in the role attribute:

The value MUST be one of the Mailbox attribute names listed in the
IANA "IMAP Mailbox Name Attributes" registry at
<https://www.iana.org/assignments/imap-mailbox-name-attributes/>,
as established in [[RFC8457](https://www.rfc-editor.org/rfc/rfc8457)], converted to lowercase.

The attribute outbox is not listed in https://www.iana.org/assignments/imap-mailbox-name-attributes/ or RFC8457.

Actually in JMAP the functionality of the Outbox folder is replaced by the EmailSubmission object. To find out whether a message was sent or pending to be delivered you could do EmailSubmission/get.

Thank you for working on this!

[chibenwa]

Hello @mdecimus

First I'm super happy to see your interest toward this application!

Tmail application is primarily thought to be used... On top of TMail backend! We of course have plans to work on inter-operability topics in the near future (ETA ~end October) so we will save these topics when we would work on getting Tmail apps working on top of FastMail.

Of course, contributions are welcome. Of course we could consider support options to accelerate the "inter-operability" part of the roadmap and assist you getting TMail app working on top of your Email server (including OIDC).

I would be more than happy to have a call with you to discuss possible future collaborations. Don't hesitate to drop an email about it (my email address is available on my profile).

The technical points you raise (ids in Email/set, client side provisionning of compulsory mailboxes, being resilient regarding "outbox" role) are very legitimate and would be addressed when we will start working on inter-operability.

Best regards,

Benoit TELLIER

[mdecimus]

Hi @chibenwa ,

Tmail application is primarily thought to be used... On top of TMail backend! We of course have plans to work on inter-operability topics in the near future (ETA ~end October) so we will save these topics when we would work on getting Tmail apps working on top of FastMail.

I totally understand. However, I believe that compliance with the JMAP Core and Mail specifications should also be a priority for your TMail backend. Deviations from the standard such as using proprietary mailbox roles may break compatibility with third-party JMAP/IMAP clients accessing the TMail backend. But since you plan to be full compliant around October then this is not a big deal.

Of course, contributions are welcome. Of course we could consider support options to accelerate the "inter-operability" part of the roadmap and assist you getting TMail app working on top of your Email server (including OIDC).

I'll see if I can send a pull request with those fixes after our release date. We have recently migrated our entire mail system to JMAP and TMail is the only client we use internally. Unfortunately we'll have to rely on Ltt.rs to send emails for the time being.

I would be more than happy to have a call with you to discuss possible future collaborations. Don't hesitate to drop an email about it (my email address is available on my profile).

Sure, I'll send you an email in a few weeks once the first version of our server is made public.

Thank you, Mauro

[chibenwa]

I'll see if I can send a pull request with those fixes after our release date. We have recently migrated our entire mail system to JMAP and TMail is the only client we use internally.

I'm really happy to hear that!

Unfortunately we'll have to rely on Ltt.rs to send emails for the time being

Hopefully we will be able to solve that matter of fact quickly!

[hoangdat]

It is including an htmlBody part with an invalid blobId and in addition to that also specifies a partId. According to RFC8621 The client may specify a partId OR a blobId, but not both.

Fixed in #921

[mdecimus]

Thanks @hoangdat

[hoangdat]

We fixed the error with send email in #942 (auto add outbox folder if missing). Maybe in one day, docker image with master tag will be built with this fix. For mobile application, may be in 2 weeks app will be available in store. If you want to try earlier, let me know, I will send you the artifacts from our CI/CD

[mdecimus]

Thank you for the update. I'll give it a try soon. Does it create the Outbox folder without using the outbox role?

[hoangdat]

Does it create the Outbox folder without using the outbox role?

yes, without outbox role. A normal mailbox called outbox :).

Moreover, we tested on your server, we found that your JMAP server also don't have Sent folder (with sent role).

[mdecimus]

yes, without outbox role. A normal mailbox called outbox :).

Excellent! Looking forward to give it a try.

Moreover, we tested on your server, we found that your JMAP server also don't have Sent folder (with sent role).

The test server at https://jmap.cloud may not have it, but if you install the JMAP server locally, mailboxes with the inbox, trash, drafts, sent and junk roles are created automatically for each new account.

Also, I found an unrelated bug that I am not sure you are aware of. I am using Tmail from my browser (Firefox) and also my phone (iPhone). I noticed that when I delete messages from one client they still appear on the other client and cannot be deleted. Even if I close the browser and reopen it, the deleted messages are still displayed. The only way to fix it is to go to Developer tools and delete all session data.

[hoangdat]

• yes, known issue, we try to solve our pain - caching problem, please try with the new release. (v0.3.15).
• We are also investigating on the problem: synchronizing between IMAP and JMAP client.

[hoangdat]

if you have further feedback, please raise in a new ticket. For better understanding for all people. We will close this ticket. Are you ok?

[mdecimus]

We will close this ticket. Are you ok?

Closed!