PHP compressed value can't process

[ghost]

Hi,

We have faced an issue when the key is stored via php with compression it can't process in memcached module.

php code: Memcache::set ( "SampleKey" , "Sample value" , MEMCACHE_COMPRESSED , 2000);

undefined:1 x����N� ^ SyntaxError: Unexpected token x at Object.parse (native) at Socket.value (/usr/local/lib/node_modules/memcached/lib/memcached.js:497:26) at Client.rawDataReceived (/usr/local/lib/node_modules/memcached/lib/memcached.js:708:51) at Client.BufferBuffer (/usr/local/lib/node_modules/memcached/lib/memcached.js:654:12) at Socket.bowlofcurry (/usr/local/lib/node_modules/memcached/lib/utils.js:108:15) at Socket.EventEmitter.emit (events.js:95:17) at Socket. (stream_readable.js:746:14) at Socket.EventEmitter.emit (events.js:92:17) at emitReadable (_stream_readable.js:408:10) at emitReadable (_stream_readable.js:404:5

[3rd-Eden]

The module doesn't support compressed values as it has no logic to unzip the compressed information.

[ghost]

Is there any option to compress.decompress the data using node memcached module instead of storing raw data ?

[rcoup]

@kazzalib unfortunately, each memcached client tends to use the type flags in the protocol differently, so what the PHP library interprets as a zipped string node-memcached might interpret as an integer and python's memcached library might interpret as a double and so on. String is the only type that is consistent in my experience.

So, one option is to pre-gzip your data (might need to base64 it too) in PHP and store it as a string to memcached? Then you'll get a string back in node and can un-gzip it.

[avin45h]

I was able to fix a similar issue. We were facing same kind of php compressed data. Have a look at the diff file below, (It may sound like a nasty fix with a lot of side effects, but works like a charm in our case). We were unable to change original php code due to backward compatibility limitations with existing php code base. Works good for both case, compressed and non compressed.

diff --git a/node_modules/memcached/lib/memcached.js b/node_modules/memcached/lib/memcached.js
index d699f2b..141fa94 100644
--- a/node_modules/memcached/lib/memcached.js
+++ b/node_modules/memcached/lib/memcached.js
@@ -4,7 +4,8 @@
  * Node's native modules
  */
 var Stream = require('net').Stream
-  , Socket = require('net').Socket;
+  , Socket = require('net').Socket
+  , zlib = require('zlib');

 /**
  * External or custom modules
@@ -176,7 +177,7 @@ Client.config = {
       S.streamID = sid++;
       S.setTimeout(memcached.timeout);
       S.setNoDelay(true);
-      S.setEncoding('utf8');
+      S.setEncoding('binary');
       S.metaData = [];
       S.responseBuffer = "";
       S.bufferArray = [];
@@ -728,7 +729,12 @@ Client.config = {

         // fetch the response content
         if (tokenSet[0] === 'VALUE') {
-          dataSet = Utils.unescapeValue(S.bufferArray.shift());
+          var value = S.bufferArray.shift();
+          try{
+            value = zlib.inflateSync(new Buffer(value, "binary")).toString()
+          }catch(e){
+          }
+          dataSet = Utils.unescapeValue(value);
         }

         resultSet = privates.parsers[tokenSet[0]].call(S, tokenSet, dataSet || token, err, queue, this);

[joule-xx]

Actually php has two different sets of encoding depending on... who knows... it will either encode with zlib or fastlz. I have worked around the decoding bit and will add soon the encoding bit. Unfortunately you will need also a fastlz module (which is used 99% of the time by my php instance).

I have this small diff for the memcache part and attached the code for the fastlz bit (adapted from a couple which can only compress)

dev_fastlz.zip

I am not pushing this as a feature request, as it was said often, every single developer uses it's own compression for memcache, and we can't put all of them in here, but it might help someone diff.txt