wolfssl icon indicating copy to clipboard operation
wolfssl copied to clipboard

Use signed variable for length calculation in SendTls13Certificate

Open ColtonWilley opened this issue 1 year ago • 3 comments

Description

Use signed variable for length calculation in SendTls13Certificate. Current code uses unsigned len which only works when decremented exactly to zero. Any mistakes in the length handling will underflow and keep looping, allocating memory until the system runs out.

Additional fix brought up by ZD 18267

ColtonWilley avatar Aug 16 '24 17:08 ColtonWilley

Prefer to keep length as unsigned if possible as all assignments I've seen are calculated with unsigned. Windows is complaining about signed/unsigned mismatch and needs to be fixed.

SparkiDev avatar Aug 19 '24 10:08 SparkiDev

So I mean we can keep it unsigned but I feel it is still far riskier. The hang two customers saw would have been prevented with this change, and in general any mistakes in length handling will result in a hang. Any mistakes in further modifications of that code could also result in a hang. Would strongly prefer we not do unsigned len > 0 loop, especially when allocating inside that loop.

ColtonWilley avatar Aug 19 '24 12:08 ColtonWilley

Retest this please.

ColtonWilley avatar Aug 26 '24 20:08 ColtonWilley