When uploading multiple files, the client does not preserve order

[interruptinuse]

Steps to reproduce

1. Select multiple files (e.g., mp3 files) in a file browser: a.mp3, b.mp3, c.mp3.
2. Drag and drop into a chat window.

Expected behaviour

Files are uploaded and displayed in selection order: a, b, c.

This behavior used to be the (undocumented?) status quo until recently, which I personally found useful.

Actual behaviour

Files are now uploaded in parallel. This is not an issue by itself, and actually noticeably improves upload speeds. During the upload, the order of files looks preserved on the uploading client (until relaunching the client), regardless of when each individual file upload actually finished. However, the order of uploads is not guaranteed anymore, and files may arrive shuffled depending on file size, e.g.: a, c, b.

This change was introduced in a very recent version of Telegram Desktop, most likely in 0dcc439ddae2d686fc4b63e521c1d5eb829bc3ef.

Operating system

Arch Linux, x86_64, Plasma 6

Version of Telegram Desktop

4.16.8

Installation source

Other (unofficial) source

Crash ID

No response

Logs

No response

[Aokromes]

it's os "issue" if you select 3 files and drag a file a will be upload 1st, if you select c file c will be upload fist.

[ghost]

This is not an OS issue. It's the fact that file uploads are delivered from the server at the time the file is done uploading (not when it's started uploading), and this order used to be deterministic. Now that multiple files are uploaded at the same time, this is no longer guaranteed.

[Aokromes]

https://github.com/telegramdesktop/tdesktop/issues/26971#issuecomment-1781162728

[ghost]

I'm not using Windows.

[Aokromes]

I'm not using Windows.

still it's the same "bug"

[john-preston]

Need to test. I was hoping I did preserve the order.

[ghost]

I've been trying to repro this, and so far it's been extremely finicky. I've only triggered the reordering issue with ungrouped MP3 uploads in a channel so far; it seems to work much better after a pause in traffic (subsequent attempts to reproduce are fruitless).

I actually haven't assembled a large enough sample size to confidently say that parallel uploads are to blame, but I don't think I remember it happening before.

Here's what it looks like:

1. In the file manager (Dolphin)

2. In the Arch Linux 4.16.8 client after uploading (correct order)

3. In mobile and web clients (wrong order)

So it does seem to me that it's an upload heisenbug.

[ghost]

Another repro with white noise MP3s.

Generator script

#!/usr/bin/env bash
set -ex
for i in {01..20} ; do
  COVER_SIZE=500
  head -c "$((3*COVER_SIZE*COVER_SIZE))" /dev/urandom \
    | convert -depth 8 -size "${COVER_SIZE}x${COVER_SIZE}" RGB:- $i.jpg

  DURATION=$(( 100 + ( RANDOM % 100 ) )).$(( RANDOM % 100 ))
  ffmpeg -f lavfi -i "anoisesrc=a=0.1:c=white:d=$DURATION" -i $i.jpg \
    -map 0:0 -map 1:0 -id3v2_version 3 \
    -metadata:s:v title="Album cover" -metadata:s:v comment="Cover (front)" \
    -metadata artist="White Noise" -metadata title="$i" -b:a 192k -c:v copy -y $i.mp3
  rm -v $i.jpg
done

Arch client, after uploading

iOS client

[UpscaleAnon]

I think it depends on size of file, because I've had this happen a lot in the past with other files. I think it's because telegram processes the files a bit before they get sent in a channel and if the second file is finished processing (not uploading) before the first one, it'll get "sent" first, no matter what you do. It is extremely annoying. Telegram seems more prone to this after the recent update that boosted upload speed (Thank fuck for that update, finally usable speeds).

[john-preston]

I couldn't reproduce that.

@interruptinuse Can you please:

1. launch the app
2. enable debug logs by typing "debugmode" in Settings (like a cheat-code in a game)
3. reproduce the bug (upload the files and see them with a wrong order in a mobile app)
4. make screenshots of uploaded files and mobile app files order
5. in Settings type "viewlogs" (like a cheat-code in a game) to see where the logs are stored
6. close the app completely (using Ctrl+Q keyboard shortcut)
7. .zip DebugLogs folder and send it to me at https://t.me/preston (this logs will contain all the network activity from the moment you've enabled them and till quit)

I'll see the send file requests order at least.

[john-preston]

I hope I've found the problem. I'm building a test version now..

[john-preston]

I hope this will be fixed in 5.0.2 later today.

[ghost]

@john-preston Could not reproduce the issue with build_5_0_1_1.tar.xz so far. I did have a few MSG_WAIT_FAILED responses in MTP logs, but no reordering issues.

[ghost]

Closing as fixed in 84ec2a5f7494ffc2516ecc78ef89aba3da199b33 (5.0.2).

[ghost]

Reordering behavior regrettably still appears in 5.1.5 (same platform).

[github-actions[bot]]

This issue has been automatically closed because no developer succeeded to reproduce the issue with the given reproduction steps. With only the information that is currently in the issue, we don't have enough information to take action. Please reach out if you find what's missing to reproduce the issue so that we can investigate further.

Note that GitHub is a developer communication platform. If you're an ordinary user seeking for help, get to support crew via Settings -> Ask question in the application.