toml icon indicating copy to clipboard operation
toml copied to clipboard

Cannot roundtrip strings with \x in them

Open dwt opened this issue 2 years ago • 3 comments

While using responses to mock a response for http://google.com, I got an error because the output from google was saved, but couldn't be read again afterwards. After some searching, I was able to reduce the problem down to this:

toml.loads(toml.dumps(dict(foo=r'b\xar')))

Original context looked more like this:

:c.body;a=d.clientWidth;b=d.clientHeight}a&&b&&(a!=google.cdo.width||b!=google.cdo.height)&&google.log(\"\",\"\",\"/client_204?&atyp=i&biw=\"+a+\"&bih=\"+b+\"&ei=\"+google.kEI);}).call(this);})();</script> <script nonce=\"nikMV0zuxjnA6AEiBQsZZQ\">(function(){google.xjs={ck:'xjs.hp.HLYI73ptMgw.L.X.O',cs:'ACT90oFhExVn_F_yx0vx9XuwWMrSg2w4sw',excm:[]};})();</script>  <script nonce=\"nikMV0zuxjnA6AEiBQsZZQ\">(function(){var u='/xjs/_/js/k\x3dxjs.hp.en.zkVLWnc1YR4.O/am\x3dAADoBABQAGAB/d\x3d1/ed\x3d1/rs\x3dACT90oEbq1dNoeg2P5W4Z_MI-NYeBnBS0A/m\x3dsb_he,d';var amd=0;\nvar d=this||self,e=function(a){return a};var g;var l=func

This seems like a bug in toml? If not, what to do to make this work?

dwt avatar Feb 03 '23 11:02 dwt

Investigating this further with other toml libraries, it seems that the string encoder of this toml library is likely at fault:

>>> import toml, tomli_w
>>> toml.dumps(dict(foo=r'b\xar'))
'foo = "b\\xar"\n'
>>> tomli_w.dumps(dict(foo=r'b\xar'))
'foo = "b\\\\xar"\n'
>>> toml.loads(tomli_w.dumps(dict(foo=r'b\xar')))
{'foo': 'b\\xar'}

dwt avatar Feb 03 '23 11:02 dwt

I don't know about the tool-spec and what is expected from it, is it that all backslashes should be escaped with another backslash? If so, I would be glad to provide a pull request as this is currently blocking my usage of this library through responses.

dwt avatar Feb 06 '23 06:02 dwt

Yes, this is #404. You should probably move away from this library. I switched to tomli and its companion tomli_w. tomli has been included in the standard library of recent Python versions as tomllib.

davidfokkema avatar Feb 20 '23 18:02 davidfokkema