jpype icon indicating copy to clipboard operation
jpype copied to clipboard

jpype#1091: allow null characters in strings when convertStrings=True

Open Christopher-Chianelli opened this issue 2 years ago • 3 comments

Previously, PyUnicode_FromString(str.c_str()) was used when convertStrings=True. If the string contains null bytes, this will terminate the string early. To allow strings with null bytes to be returned, PyUnicode_FromStringAndSize(str.c_str(), str.length()) is now used instead, which create a Unicode object from the pointer with the given length (instead of the position of the first NULL character), thus allowing null bytes in the string.

Fixes #1091

Christopher-Chianelli avatar Sep 16 '22 20:09 Christopher-Chianelli

You should add tests using a mixture of Nulls with international characters (16 and 32 byte encoded) to make sure that it properly handles all cases.

Thrameos avatar Sep 16 '22 20:09 Thrameos

Codecov Report

Merging #1092 (9694d43) into master (82a7658) will increase coverage by 0.02%. The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #1092      +/-   ##
==========================================
+ Coverage   88.64%   88.66%   +0.02%     
==========================================
  Files         112      112              
  Lines       10247    10247              
  Branches     4012     4012              
==========================================
+ Hits         9083     9086       +3     
+ Misses        705      703       -2     
+ Partials      459      458       -1     
Impacted Files Coverage Δ
native/common/jp_stringtype.cpp 94.28% <100.00%> (ø)
native/common/jp_encoding.cpp 75.65% <0.00%> (+2.60%) :arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

codecov[bot] avatar Sep 16 '22 20:09 codecov[bot]

Done; included tests that test strings with null bytes that SHOULD be 16 byte encoded/ 32 byte encoded.

Christopher-Chianelli avatar Sep 16 '22 20:09 Christopher-Chianelli

LFTM

Thrameos avatar Oct 26 '22 19:10 Thrameos

Thanks for fixing this and extending the test!

marscher avatar Oct 27 '22 15:10 marscher