Parallel transfer breaks if Matrix media repo doesn't include Content-Length header

[bttmchnd]

tamara_1        | [2020-01-27 03:10:46,661] [[email protected]] Startup actions complete in 0.77 seconds, now running forever
tamara_1        | [2020-01-27 03:11:16,289] [[email protected]] Received event: MessageEvent(type=EventType("m.sticker", EventType.Class.MESSAGE), room_id='!mQDlWhEQkHYVqWaxdV:sunbutt.faith', event_id='$p64XaimoTam_Yghj9qk77GZSG4b5z20JaYQosak7W8A', sender='@commagray:sunbutt.faith', timestamp=1580094676215, content=MediaMessageEventContent(msgtype=m.sticker, body='😂', external_url=None, _relates_to=None, url='mxc://sunbutt.faith/EDWgAUNLivYOlyDSaRdGxaeR', info={'dimension': {'author': {'name': 'Telegram', 'reference': 'https://t.me/addstickers/DevStickersPack01', 'type': 'telegram'}, 'license': {'name': 'Telegram', 'urlPath': '/licenses/telegram-imported.txt'}}, 'h': 256, 'mimetype': 'image/png', 'thumbnail_info': {'h': 256, 'mimetype': 'image/png', 'w': 256}, 'thumbnail_url': 'mxc://sunbutt.faith/EDWgAUNLivYOlyDSaRdGxaeR', 'w': 256, '__mautrix_msgtype': 'm.sticker'}), unsigned=MessageUnsigned(age=63, transaction_id=None))
tamara_1        | [2020-01-27 03:11:16,300] [[email protected]] Received Matrix event "MediaMessageEventContent(msgtype=m.sticker, body='😂', external_url=None, _relates_to=None, url='mxc://sunbutt.faith/EDWgAUNLivYOlyDSaRdGxaeR', info={'dimension': {'author': {'name': 'Telegram', 'reference': 'https://t.me/addstickers/DevStickersPack01', 'type': 'telegram'}, 'license': {'name': 'Telegram', 'urlPath': '/licenses/telegram-imported.txt'}}, 'h': 256, 'mimetype': 'image/png', 'thumbnail_info': {'h': 256, 'mimetype': 'image/png', 'w': 256}, 'thumbnail_url': 'mxc://sunbutt.faith/EDWgAUNLivYOlyDSaRdGxaeR', 'w': 256, '__mautrix_msgtype': 'm.sticker'})" from @commagray:sunbutt.faith in !mQDlWhEQkHYVqWaxdV:sunbutt.faith
tamara_1        | [2020-01-27 03:11:16,313] [[email protected]] 192.168.240.3 [27/Jan/2020:03:11:16 +0000] "PUT /transactions/3170?access_token=REDACTED HTTP/1.1" 200 158 "-" "Synapse/1.9.0"
tamara_1        | [2020-01-27 03:11:16,336] [[email protected]] Exception in Matrix event handler
tamara_1        | Traceback (most recent call last):
tamara_1        |   File "/usr/lib/python3.8/site-packages/mautrix/appservice/appservice.py", line 229, in try_handle
tamara_1        |     await handler_func(event)
tamara_1        |   File "/usr/lib/python3.8/site-packages/mautrix/bridge/matrix.py", line 319, in int_handle_event
tamara_1        |     await self.handle_message(evt.room_id, evt.sender, evt.content, evt.event_id)
tamara_1        |   File "/usr/lib/python3.8/site-packages/mautrix/bridge/matrix.py", line 237, in handle_message
tamara_1        |     await portal.handle_matrix_message(sender, message, event_id)
tamara_1        |   File "/usr/lib/python3.8/site-packages/mautrix_telegram/portal/matrix.py", line 389, in handle_matrix_message
tamara_1        |     await self._handle_matrix_file(sender_id, event_id, space, client, content, reply_to,
tamara_1        |   File "/usr/lib/python3.8/site-packages/mautrix_telegram/portal/matrix.py", line 254, in _handle_matrix_file
tamara_1        |     file_handle, file_size = await parallel_transfer_to_telegram(client, self.main_intent,
tamara_1        |   File "/usr/lib/python3.8/site-packages/mautrix_telegram/util/parallel_file_transfer.py", line 298, in parallel_transfer_to_telegram
tamara_1        |     return await _internal_transfer_to_telegram(client, response)
tamara_1        |   File "/usr/lib/python3.8/site-packages/mautrix_telegram/util/parallel_file_transfer.py", line 266, in _internal_transfer_to_telegram
tamara_1        |     part_size, part_count, is_large = await uploader.init_upload(file_id, file_size)
tamara_1        |   File "/usr/lib/python3.8/site-packages/mautrix_telegram/util/parallel_file_transfer.py", line 199, in init_upload
tamara_1        |     connection_count = connection_count or self._get_connection_count(file_size)
tamara_1        |   File "/usr/lib/python3.8/site-packages/mautrix_telegram/util/parallel_file_transfer.py", line 135, in _get_connection_count
tamara_1        |     if file_size > full_size:
tamara_1        | TypeError: '>' not supported between instances of 'NoneType' and 'int'

But they work from Telegram to Matrix.

Version: latest Docker image.

[a22sc]

Getting a similar behavior with a pdf file

[2020-10-13 12:00:36,051] [[email protected]] Exception in Matrix event handler 
Traceback (most recent call last): 
  File "/opt/mautrix-telegram/lib/python3.8/site-packages/mautrix/appservice/as_handler.py", line 157, in try_handle 
    await handler_func(event) 
  File "/opt/mautrix-telegram/lib/python3.8/site-packages/mautrix/bridge/matrix.py", line 417, in int_handle_event 
    await self.handle_message(evt.room_id, evt.sender, evt.content, evt.event_id) 
  File "/opt/mautrix-telegram/lib/python3.8/site-packages/mautrix/bridge/matrix.py", line 278, in handle_message 
    await portal.handle_matrix_message(sender, message, event_id) 
  File "/opt/mautrix-telegram/mautrix_telegram/portal/matrix.py", line 369, in handle_matrix_message 
    await self._handle_matrix_message(sender, content, event_id) 
  File "/opt/mautrix-telegram/mautrix_telegram/portal/matrix.py", line 416, in _handle_matrix_message 
    await self._handle_matrix_file(sender_id, event_id, space, client, content, reply_to, 
  File "/opt/mautrix-telegram/mautrix_telegram/portal/matrix.py", line 255, in _handle_matrix_file 
    w, h = content.info.width, content.info.height 
AttributeError: 'FileInfo' object has no attribute 'width'

on latest master branch

[tulir]

@a000a that's a completely unrelated problem, fixed in fdc58ce450f3768bc9cbef4d62fde130dde8ef05

[tulir]

So the problem here is that the parallel transfer option only works if the media repo includes a Content-Length header. I think matrix-media-repo doesn't include it in some cases, Synapse probably always does. A temporary fix is to disable parallel transfers in the config. The proper solution is to automatically fall back to normal file transfer when there's no content length header.

[Ugzuzg]

Seems to be breaking stickers (Matrix -> Telegram), too:

First one was sent with parallel_file_transfer: true, second with parallel_file_transfer: false.

[tulir]

That's expected, the bridge can't convert images to webp without having the whole thing in memory. If the stickers are in the correct format already on Matrix, then they should work on Telegram too.

[Ugzuzg]

All the default dimension stickers use PNG, and it even converts the stickers imported from Telegram to PNG. Will you accept a PR that ignores parallel_file_transfer setting for stickers?