Wide character problem

[LittleFox94]

Hi,

we are using this package to cache translations in our web application. They often contain wide characters, which is working fine when no compression is enabled. Once we enable compression (tested with LZ4 and Snappy), storing something in the Cache crashes with "Wide character in subroutine entry at [..]/Cache/FastMmap.pm line 740".

Since wide characters are OK when not compressing, I would think handling wide characters would be the responsibility of Cache::FastMmap and not the caller.

What do you think of it?

A patch could be supplied if you think the same way :)

Thanks and best regards

[LittleFox94]

Maybe this is already fixed by not using raw_values => 1. I let this ticket open because I don't know if this has other side effects or worse performance.

[robmueller]

This is definitely a bug. Currently it checks if the stored value has the UTF8 flag set in the C code, but it does the compression in perl. The compression library won't like if you pass a string with the UTF8 flag set.

So for the moment, change raw_values => 0 and it will work, because it uses Storable to serialise the data. A bit slower, but should be fine.

The correct solution is to check and remove the UTF8 flag in perl before compressing or passing to the C layer. It's a bit ugly because you probably want to turn it back on at the end again so the string you passed in isn't corrupted.