Using 10.0.4: exports only a fraction of messages (to EML format)

[bjd-pfq]

Subject line says all. I am using 10.0.4 with Thunderbird 68.10.0 (64-bit) on Linux.

I have a mailbox of 9481 messages; it only exports 41. Another mailbox of 449 messages: only exports 69.

I am using the defaults, only have set the file date format to: %Y%m%d-%H%M%S and maybe checked "add date to the file properties" (I don't know what the default was).

Surely I'm not the only one seeing this rather severe bug....??

[bjd-pfq]

Here's a weird observation:

I have a mailbox with 9481 messages. I am exporting 'all files in this folder to EML format'.

When I have the custom date for filenames as described above, %Y%m%d-%H%M%S, the addon exports 41 messages.

When I use the default date, it exports 499 messages out of 9481, but the red button-like indicator stays visible, i.e. the addon is still working, although the number of files stays at 499. I thought compacting the folder might be an issue. So I compacted. Nope, still exports just 499. At least it's consequential in its failings.

I noticed 9481 / 499 = exactly 19 ...! Probably just coincidence (?) but why is it exporting 41 with a custom date, 499 with the default date, but never the one and only correct number of 9481?

[bjd-pfq]

Somebody please look into this and give me some feedback here -- this is a severe bug that may cost people their emails.

[bjd-pfq]

When I choose HTML instead of EML all symptoms as described above remain.

[cleidigh]

@bjd-pfq thanks for posting, sorry for the issues,we can do some back-and-forth to diagnose. unfortunately there are so many permutations for problems, but hopefully we can track it down.

First: can you check the console for errors and post here. also can you describe the structure of the import there was a bug on Windows when there were to many files open the 499 number is suspicious because that's the max file number and I have address that a while ago. there are also scenarios where file structure has issues hopefully you will be able to help as I cannot usually reproduce errors that are based on the particulars of someone's e-mail so let's see if we have any clues in the log (Control-shift-J) you can select all and post here Christopher happy New Year's!

[bjd-pfq]

Thanks for responding.

I am getting:

myInbox exportTools.js:359:10 NS_ERROR_FILE_NAME_TOO_LONG: Component returned failure code: 0x80520011 (NS_ERROR_FILE_NAME_TOO_LONG) [nsIFile.createUnique] exportTools.js:1147 onStopRequest68 chrome://mboximport/content/mboximport/exportTools.js:1147

This is with my custom date format as described above (only 41 emails out of 9481 are exported as EML).

The error makes me wonder if the following might be the limiting factor: I am working on a Linux Mint system, to be precise: on an encrypted home directory (ecryptfs). Ecryptfs, when it also encryptfs filenames (mine does) has a filename limit of I believe 144 characters.

If this is the limiting factor, or if not, in general, I think IE-NG should issue a warning that not all files are exported; it should most definitely NOT silently fail. I'm betting people have already lost emails because of this.

[bjd-pfq]

When I limit the Subject to 50 characters 7193 out of 9483 are exported. You see the trouble with silently failing? (Oh, and the red indicator persists).

[bjd-pfq]

When I uncheck "Customized format for filenames" AND "cut subject at 50 chars" it exports all messages and creates an index.html file (I had never seen that, nor did I know it would be created if all went well) But who is to say if at 10,000 messages, or 20,000, the addon will fail?

[bjd-pfq]

Also, I have set charset for filenames to 'utf-8'; doesn't that place an extra limit on filenames??

[bjd-pfq]

Different issue, but can we have a more European date (or ISO) in the index.html? The American style date is non-standard imho, ambiguous and generally ugly as hell. But first and foremost, the addon has to be made robust, and issue warnings, errors and progress reports that are clearly visible to the user.

[bjd-pfq]

My /tmp is located on an unencrypted drive, and when I use my custom date (see above) and do NOT limit the subject to 50 characters, all messages are exported to a destination in /tmp.

So the underlying filesystem NAME_MAX is the limiting factor. The addon might first do some heuristics to see what the actual filename limit is for the destination the user has chosen, and issue a NO GO Warning when the user's settings conflict with this.

Again, silently failing is being lethal & hiding the bodies.

[bjd-pfq]

Actually I misspoke. The ecryptfs limit seems to be 143, see: https://unix.stackexchange.com/questions/32795/what-is-the-maximum-allowed-filename-and-folder-size-with-ecryptfs/32834#32834 But also see the important comment on UTF-8 encoded filenames.

[cleidigh]

@bjd-pfq you are absolutely correct the errors need to be surfaced. since taking over for extension I have been slowly patching various errors however most of my work has usually been on the import side. a prescan approach used for the import allows detecting certain issues before the import. I have to do the same thing on the export side. the scenario of encrypted folders is something not at all contemplated by the extension. while I do not think it is a requirement it might be useful for me to have a test setup on my Mint VM how difficult is it to set up? the problem is being disabled I have to do everything by voice and remotely with the VM - anything command line is a PIA

concerning your comment about the index date format, if you look at the help IRG have an option to customize the index date 'it's just the UI yet as described in the help: I will post progress here... Christopher