No clear error thrown when media larger than 10mb is uploaded

[nike-hercules]

I have a bot that posts fairly large images (up to around 4 MB). First it ran fine, but over time more and more regularely project.post() fails with following error message, up to the point that it now fails every time:

Traceback (most recent call last):
  File "/home/me/.local/lib/python3.11/site-packages/cohost/network.py", line 61, in fetch
    res = req.json()
          ^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/requests/models.py", line 897, in json
    return complexjson.loads(self.text, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/me/SlideshowBot/bot.py", line 26, in <module>
    newPost = project.post(
              ^^^^^^^^^^^^^
  File "/home/me/.local/lib/python3.11/site-packages/cohost/models/project.py", line 226, in post
    attachment.uploadIfNot(req['postId'], self)
  File "/home/me/.local/lib/python3.11/site-packages/cohost/models/block.py", line 99, in uploadIfNot
    dospacecreds = fetch('postjson', endpoint, {
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/me/.local/lib/python3.11/site-packages/cohost/network.py", line 63, in fetch
    raise Exception(req.text)
Exception

The relevant code is

user = User.login(user, pw)
project = user.getProject(projectName)
blocks = [
    AttachmentBlock(imagePath),
]
newPost = project.post(
    "", blocks, tags=[tagList]
)

I have two other bots (one that posts small images and one that posts text only) that run without issues with basically the same code.

[valknight]

hm, this is odd!

As for what I know is going on from this stacktrace - Cohost.py assumes whatever being sent back over the API is encoded in JSON. So, this error occurs when Cohost is sending some non-JSON response over the API back to us. In the past this was seen when we got hit with Cohost's DDOS protection (at the time being Cloudflare) blocking Cohost.py - changing the user agent fixed that though.

My guess (without knowing what's in the failing request...) is that something similar is happening, but only triggering when uploading lots of data.

Interestingly though, this is looking like Cohost is sending... nothing back. We raise an exception, with what should be the contents of the response, and, nothing is in the console. Quick sample showcasing this is probably an empty string being passed, using the Python interpreter.

In this example, note : demo!" after Exception on the last line

Type "help", "copyright", "credits" or "license" for more information.
>>> raise Exception("demo!")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
Exception: demo!

In this example, not there is no string, and no semicolon

>>> raise Exception("")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
Exception
>>>

Also interestingly, this is failing when we're requesting credentials to upload, and isn't failing at the step we're uploading images. As such, I'm really curious why Cohost isn't sending any data back. My best guess is that it sends nothing back when the 'content_length' is too high.

Just as I'm a little busy having some kind of cold / sickness (I'm mostly typing this looking away from the laptop screen, because the glare is Not It 😭 ), could you let me know:

• what do you see when you attach a debugger, and then add a breakpoint on line 63 of network.py?
• does this reproduce 100% with the same input? ie: some images simply cannot be uploaded, while others can
  • if not, how often does this reproduce with the same input?
  • if you need to test this without spaming people, I'd recommend making a private page, and then uploading posts there

thanks for raising an issue, and thanks for using cohost.py!

[nike-hercules]

Thanks for your response! As it turns out, the whole thing was entirely my fault: The bot randomly went through a folder of pictures to post, then removing them. There were, however, a few pictures with a size >10MB. As there were less other pictures to choose from, more and more are oversized until only those were left... So the bug is i guess the inability to handle the response for too large attachment files. Again, thanks for your help, for making the library & get well soon!

[valknight]

No worries! I'll leave this open, just rename to be an FR for clearer errors when using images bigger than 10mb.