Does not work with TB 115

[m707]

I tried the most recent build, but after hitting connect I just get:

An unexpected error occurred

:(.

[chris21k]

Does not work with the TB 115.0 release version.

[creative-coder]

try this, its a pached version and works... https://trella.info/allgemein/thunderbird-sieve-filter-funktioniert-nicht/

[m707]

Not for me in TB 116.0b5 (updated in the meantime):

Connection to server failed :(, ... or in our $LANG: "Verbingung zum Server fehlgeschlagen ;). An unexpected error occured.

Are you sure you got that version working with a TB version >= 115?

[creative-coder]

my mistake, im running 102.13.0.. but works at this version (sry was late tonight)

[vsatmydynipnet]

Found out in https://github.com/thsmi/sieve/pull/768 too. too. Looks like we need help of a dev to overcome this and be able to use 115.

[thsmi]

There was a pretty massive and breaking change in Mozilla's socket implementation. Refer to the following bug for more details: https://bugzilla.mozilla.org/show_bug.cgi?id=1791633

Beside this the following change also needs to be addressed https://bugzilla.mozilla.org/show_bug.cgi?id=1780695

And of course the other bugs which are blocking the 102 release...

So don't expect a release anytime soon, unless the unlikely happens and someones steps up and provides fixes...

Until then the only suggestion I can give is to use the standalone app.

[sachaz]

Thanks for the reply & letting us know whats going on <3

[jobisoft]

Hi,

the drop of Services.jsm is not much work: we do not have to include it at all anymore. It is now a globally available variable.

For the socket, I would like to move away from the internal implementation and use a JavaScript library. But I need a server to test against. Is anybody here able to provide me a test account/access? You can reach out to me directly via [email protected]

Thanks!

[jobisoft]

@thsmi

I have not understood why the WebExtension cannot use the same trick you use in the Web App, to connect to the server, or why the WebSocket API cannot be used.

I must be missing something crucial and would be happy if you could help me to understand the situation better. If your new Web App is based on node.js, should you not be able to webPack it into a Firefox/Thunderbird extension?

[thsmi]

@jobisoft well it is for the same reason why Thunderbird does not use the WebSocket API for IMAP, POP3 or SMTP. Those protocols are like SIEVE based exclusively on TCP not on HTTP. Thus WebSockets can't be used, they work only with HTTP Servers.

Thunderbird does not provide an API for extension to create TCP connections. So you need to use an experiment to open TCP Sockets. But every major release introduces a breaking change to the Socket API. And because the socket API is completely incompatible to the node.js socket implementation webpack don't work.

The WebApp works like a webmailer which talks to you browser in http while it talks to your mailserver in imap and smtp or respectively SIEVE. It is basically a WebSocket to SIEVE translator or proxy. The WebApp provides both the http server and the proxy. This allows a webbrowser to magically communicate with a SIEVE endpoint.

But this does not work with Thunderbird, because it is a client and not a server. Thus the only option to use websockets is that you have somewhere a server where you could run such a WebSocket to SIEVE proxy. Thus the only real world option is to use a very classic socket connection. And this is why you have to use an Experiment.

And yes you could try to standardize Sieve over WebSocket. But then the hard part starts, how to convince the mail server implementations to implement this? There is no use-case other then Thunderbird for this. Normal Applications just use a standard socket connection and WebApp anyhow need an HTTP Server to provide the app to the browser and thus you can easily provide a proprietary websocket to sieve proxy as side channel....

I guess you see why Thunderbird is so special. It officially offers no sockets support to extension, only HTTP or WebSockets. And Mailservers typically don't provide HTTP or Websockets endpoints for security concerns. Without Experiments there is no overlap and the Extension would be long dead.

The real pain point here is that Mozilla introduces with each major release a breaking change for their socket implementation. I don't know why but it seems to be a long standing tradition. Node.js in contrast provides a rock solid API for TCP Sockets. there haven't been any breaking changes since many many years. But this is something which can't be changed...

[madduck]

Does anyone know of any progress on this?

[ecogit]

Is there a way to donate as a token of gratitude for the efforts by @thsmi and his fellow contributors? ☕️

[madduck]

See here: https://github.com/thsmi/sieve#status

I've offered to @thsmi that my company could pay him to get sieve up to speed and am hoping to get a reply soon.

[thsmi]

Found some time to look into this and prepared a patch.

But keep in mind it is not really tested. It seems to connect but could fail at other spots And the drag in drop in the graphical editor is still badly broken in Thunderbird. Which makes it unlikely that there will be an official release soon.

You can get a nightly build from this link. https://dev.azure.com/thsmi/1164eec3-b870-4973-8110-5e275d208aac/_apis/build/builds/19977/artifacts?artifactName=XPI%20-%20Thunderbird%20WebExtension&api-version=7.1&%24format=zip

This build does not work with thunderbirds daily builds. Because of the following bug: https://bugzilla.mozilla.org/show_bug.cgi?id=1851294 . It will instantaneously crash Thunderbird Daily Release when starting the TLS session.

[madduck]

I gave it a bit of a whirl, and it runs very nicely. Drag and drop works too, or at least I have not run into any problem thus far.

Thank you! I've just sent over a little donation. Get in touch at mail ät martin-krafft dot net if you want to explore options to fund an official release…

[blacklion]

Nightly build works for me in 115, thank a lot! I'm using only text editor (not visual one), so can not report status of drag'n'drop

Thank you again.

[tuxick]

Found some time to look into this and prepared a patch.

But keep in mind it is not really tested. It seems to connect but could fail at other spots And the drag in drop in the graphical editor is still badly broken in Thunderbird. Which makes it unlikely that there will be an official release soon.

Great, works for me on 115.4.1. Thanks a lot.

[elfloo]

Found some time to look into this and prepared a patch.

But keep in mind it is not really tested. It seems to connect but could fail at other spots And the drag in drop in the graphical editor is still badly broken in Thunderbird. Which makes it unlikely that there will be an official release soon.

You can get a nightly build from this link. https://dev.azure.com/thsmi/1164eec3-b870-4973-8110-5e275d208aac/_apis/build/builds/19977/artifacts?artifactName=XPI%20-%20Thunderbird%20WebExtension&api-version=7.1&%24format=zip

This build does not work with thunderbirds daily builds. Because of the following bug: https://bugzilla.mozilla.org/show_bug.cgi?id=1851294 . It will instantaneously crash Thunderbird Daily Release when starting the TLS session.

Didn't manage to catch your link up, will be glad if you'll be able to share fresh sieve webext build again, current link shows that build cant be found.

Big thank you in advance for your time and work!

[SuperTux88]

@elfloo I still had the file I downloaded in October in my downloads folder: XPI - Thunderbird WebExtension.zip

[adlerweb]

Instead of using a potentially outdated XPI you can also download the latest nightly as described in the FAQ:

1. Head over to https://dev.azure.com/thsmi/sieve/_build?definitionId=5
2. Click on the latest (successful) build
3. Click on Jobs→Web Extension
4. Click on "1 artifact produced"
5. A "XPI - Thunderbird WebExtension" should be published. Open the menu at the right side and choose "Download artifacts"

Keep in mind: These builds are untested and might contain severe bugs.

[ploink]

@elfloo I still had the file I downloaded in October in my downloads folder: XPI - Thunderbird WebExtension.zip

Thanks! I tried several versions, including the latest nightly but only this one worked. Others do not load the server configuration.

[elfloo]

@elfloo I still had the file I downloaded in October in my downloads folder: XPI - Thunderbird WebExtension.zip

God bless you! Will try out a bit later, you're a lifesaver

[TonyGravagno]

The latest build from 13-Dec-2023 hangs on the Connecting process. As seen in the mention from #891 I'm trying to shift to using the Windows App while the WebExtension in Thunderbird is getting attention, but without config info that is difficult.

[thsmi]

To avoid confusion the patches which try to restore thunderbird compatibility are only available on the branch linked to this ticket https://github.com/thsmi/sieve/tree/893-does-not-work-with-tb-115

Means only nightly builds which are triggered by a change on this branch will contain any kind of patch regarding Thunderbird 115 compatibility.

Concerning Thunderbird 115+ compatibility issue and why it crashed on Thunderbird nightly builds there are some updates. The problem is that the Thunderbird core changed. You can trigger a startTLS only from the network thread otherwise it will crash. At the same time java script can only execute stuff on the main thread otherwise Thunderbird will crash. So it is a self-fulfilling prophecy always ending in a crash.

Technically it would have been a dead end for the extension but there are good news at this point. SMTP and other protocols are reimplemented in JavaScript by the Thunderbird developers. Which means they need as a base a maintained and reliable Socket implementation. Which can be reused by this extension.

So I started an experiment I changed the backend to this TCPSocket implementation. And it was way less work than expected.

The link to a nightly build which supports TB115+ is below: https://dev.azure.com/thsmi/1164eec3-b870-4973-8110-5e275d208aac/_apis/build/builds/20027/artifacts?artifactName=XPI%20-%20Thunderbird%20WebExtension&api-version=7.1&%24format=zip

The zip files for nightly builds are deleted on Azure DevOps after 6 weeks here is also the zip file: XPI - Thunderbird WebExtension.zip

Keep in mind when testing it, it still is very rough and will contain bugs.

While this is some kind of a step towards a release. The big show stopper is still the broken broke graphical editor. This is work in progress and there is no ETA when it will be fixed..

[tessus]

While this is some kind of a step towards a release. The big show stopper is still the broken broke graphical editor. This is work in progress and there is no ETA when it will be fixed..

I might be out of line here, but let me suggest something (while explaining how and why I came to that idea).

I loved the TB extension for one reason: I was able to edit my sieve filters directly from my mail app without having to SSH into my server. I never cared for the graphical editor. In fact I have never even tried to use it. I only use the Source tab in the app.

I do understand that the graphical editor was meant to be a major reason for this app/extension. I don't have a problem to use the standalone app, but using it from within Thunderbird is just a lot more convenient.

How about you get rid of all your headaches by removing the graphical editor from the web extension? Currently there is no compatible version on the add-on website, and you refer to the standalone app anyway. Just call the web extension a "light" version of the standalone app and add a note to the extension to use the standalone app for that psrticular feature.

I believe this is a viable solution and IMO it makes perfect sense to use a "light" web extension.

[thsmi]

The sentence you quoted was phrased a bit sub-optimal, and leaves room for interpretation thus some clarification.

The show stopper number one for a TB115 release is, that you crash TB when you start a secure socket connection from javascript. The only fix is to migrate the implementation to TCPSocket API. First step, a prototype, is done but it is still very very rough and a long way to go. And this is work in progress but there is no ETA when it is fixed. It also depends e.g. if and when TCPSockets are exposed directly to webextension.

... and of course there is the graphical editor issue where a light extension version would be debatable option. But concerning this no work on this really started.

[tessus]

Thanks for the clarification.

The only fix is to migrate the implementation to TCPSocket API. First step, a prototype, is done but it is still very very rough and a long way to go. And this is work in progress but there is no ETA when it is fixed.

Is that the zip file you posted on Dec 28th? It works on my installation. Haven't had any issues with it yet. (I can't change the port, but I use the standard port anyway.) Personally I wouldn't mind to just use a beta version until all the kinks have been resolved. I believe it is even possible to mark add-ons as beta or pre-release, if you want to expose more people to it and have them send you bug reports.

there is the graphical editor issue where a light extension version would be debatable option.

I think it's better to have a light version than none at all. Since I am not using the graphical editor, this is of course subjective. But maybe you can start a poll to gauge the public opinion.

[thsmi]

Concerning the zip from Dec 28th, yes it was just a quick hack to test if it is feasible. It worked out way better than expected. Therefore I published it. As good as it may look, it does not care about any kind of a errors. Means in case you run into any kind of connection related errors you won't get useful error messages and the extension stalls in a partly connected and disconnected state. The exceptions are either swallowed or are not caught. And fixing and cleaning this up needs lost of time the classic 80/20 rule...

Beside this I started in January at Thunderbird the proposal to expose the TCPSocket API directly to Addons. Th whole process moves due to lack of time and resources way slower than wanted. I promised a proposal for exposing it to Extensions which is still work in progress and this proposal will most likely will change several times before it the hopefully gets stable and integrated into Thunderbird. If this lands the extension does not need to use experiments anymore to implement features. Which should make it more robust for in the future.

But this means it is the classic dilemma short hack vs long term solution. From a short term view, which focuses only on the WebExtension, it would be just stick with experiments and use The TCPSocket API from the core. And then whenever the WebExtension API for socket lands, redo the work and switch everything to this API. The long term view is to first focus on getting a stable API into Thunderbird and then update the Extension to this update. So that work does not need to be done twice.

And as especially the resource time is limited, only the second option seems to be viable to me. Trying to get a stable Socket API into Thunderbird WebExtension API and then investing time to use this from the Extension.

When the Extension has again a reliable base, it would make sense to release a light version. But this is not too likely to happen soon, because it requires a ton of work to be completed...

[sachaz]

Thanks very much for all you efforts to make it possible <3 <3 <3 As we can see: Thunderbird is not really careful on all the community of plugins developers /o\

[tessus]

@thsmi I am grateful and happy to use the zip file until then. It is more than sufficient to quickly make a change in your sieve file. I only kindly ask that you create one more/last zip file where the add-on uses a big compat version number in the manifest, so that it doesn't stop working when the next (and the next, ...) TB release lands.