stdout/stderr truncation removes the significant part

[natefoo]

We truncate the stdout/stderr stored in the database at 32 KB, so this means that the end is lost. If there's a problem, the end is probably far more relevant than the beginning. Alternatively, it may make the most sense to cut out the middle and keep the first 16 KB and last 16 KB.

[jmchilton]

We do attempt to include the end:

def shrink_string_by_size(
    value, size, join_by="..", left_larger=True, beginning_on_size_error=False, end_on_size_error=False
):
    if len(value) > size:
        len_join_by = len(join_by)
        min_size = len_join_by + 2
        if size < min_size:
            if beginning_on_size_error:
                return value[:size]
            elif end_on_size_error:
                return value[-size:]
            raise ValueError(f"With the provided join_by value ({join_by}), the minimum size value is {min_size}.")
        left_index = right_index = int((size - len_join_by) / 2)
        if left_index + right_index + len_join_by < size:
            if left_larger:
                left_index += 1
            else:
                right_index += 1
        value = f"{value[:left_index]}{join_by}{value[-right_index:]}"
    return value

I wonder what is happening here - odd.

[natefoo]

Oh, sorry - I will try to dig into this more then to get an actual example. The one I have we already cleaned up so I can't compare with the tool_stdout and tool_stderr files on disk.

[jmchilton]

Just to be clear - I'm not saying it isn't a problem - it is possible like this logic is happening but the database column doesn't match the expectation of this output. Like the database is using twice as many bytes as we expect to store the data and so half of it get truncated or something like that.