Truncate large payloads to ensure we don't drop them

[rivkahstandig3636]

[DZPM]

Still happening.

Our work-around is to check the size request.body. If greatear than 1 MB, we drop the traceback.

[AwaishK]

@DZPM while reporting error on rollbar, I am also getting below error. errors on rollbar are being reported at different places in different files, do you know how can I trace back where this is coming from? And how to stop this error to pop up again?

Failsafe from pyrollbar: payload too large. Original payload may be found in your server logs by searching for the UUID.

Below is how I report errors on rollbar, where do you check the request.body size? rollbar.report_exc_info()

[DZPM]

@AwaishK I'm afraid don't have access to that codebase anymore. I guess you should check the size before reporting to rollbar. Good luck!

[camden-lowrance]

Any workaround for this? I am getting this regularly when sqlalchemy throws an exception.

[bsmedberg-xometry]

Xometry is also experiencing this problem: the data that is too large is locals data from the stack frames of the exception: it would be perfectly acceptable, if locals data is too large, to partially or complete omit it from the payload.

[brianr]

Here is a blunt instrument as a workaround... you can disable locals collection entirely with:

rollbar.SETTINGS['locals']['enabled'] = False

Agree this should be handled better; @bsmedberg-xometry 's suggestion makes sense to me.

[saiyalamarty]

We had a similar issue where the default rollbar shortener transform wasn't shortening parts of our payload. We added a custom payload handler using Link to shorten the payload with custom logic.

[schmave]

Adding this to my Rollbar config solved the problem for us:

        'shortener_keys': [
            ('request', 'POST', '*'),
            ('request', 'POST', '*', '*'),
            ('request', 'POST', '*', '*', '*'),
            ('body', 'trace', 'frames', '*', 'locals', '*', '*'),
            ('body', 'trace', 'frames', '*', 'locals', '*', '*', '*'),
        ]

The default shortener keys defined here don't truncate values inside local variable dictionaries, and they don't truncate POST values inside dictionaries either.