some key occur error during another key using pymemcache

[JungHoon1039]

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 2070, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1515, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.6/dist-packages/flask_cors/extension.py", line 161, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
  File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1513, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1499, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
  File "/home/piclick/piclick.sirs/handler/similarSearch.py", line 333, in similar_search_v2_device
    sol_info = memcached.get_shop_info(au_id, "RECO", svc)
  File "./cache/Memcached.py", line 110, in get_shop_info
    shop_info = self.client.get(shop_key)
  File "/usr/local/lib/python3.6/dist-packages/pymemcache/client/base.py", line 535, in get
    return self._fetch_cmd(b'get', [key], False).get(key, default)
  File "/usr/local/lib/python3.6/dist-packages/pymemcache/client/base.py", line 917, in _fetch_cmd
    prefixed_keys)
  File "/usr/local/lib/python3.6/dist-packages/pymemcache/client/base.py", line 883, in _extract_value
    key = remapped_keys[key]

function : Memcached.py - chk_mcc , exception : b'94'
key : 88

function : Memcached.py - get_mcc , exception : b'88'
key : 94   

    def get_mcc_data(self, p_key):
        if self.client:
            try:
                mcc_rst = self.client.get(p_key)
                if type(mcc_rst) is bytes:
                    mcc_rst = mcc_rst.decode("utf-8")
                if mcc_rst is not None and json.loads(mcc_rst) is not None:
                    matching = dict(json.loads(mcc_rst))
                    return matching
                else:
                    return None
            except Exception as e:
                print("function : {} - {} , ".format(os.path.basename(__file__), sys._getframe().f_code.co_name),
                      "exception : " + str(e))
                print("pkey : " + p_key)
                date = str(datetime.now().strftime('%Y%m%d'))
                redis.incr('CNT_GET:' + date)
                redis.incr('CNT_GET:TOTAL')
                return None
        else:
            return None

There is some exception or error occur during pymemcache running

I feel confused because some key occur error another key using pymemcache

Can the problem occur even though the pymemcache uses multi-threads?

[jparise]

pymemcache clients don't support concurrent, multithreaded usage out-of-the-box. They don't do any internal locking, etc. Only the client pool is thread-safe. If you are accessing a client object from multiple threads, you'll need to perform your own high-level locking.

See also the discussion in #195.

[JungHoon1039]

@jparise Thank you for your reply

Your answers and references really helped a lot to me

But I don't quite understand about teh exact situation because It hasn't been long since I started programming

I wonder what thread is like when accessing a server other than a client

Also, is confirming thread-safe the only way to resolve the error?

Thanks again for your help

[jparise]

I can't give a complete answer without knowing more details about your code, but generally:

1. You can share a pymemcache client across multiple threads if you add your own locking. In your example code, you could add a lock around the self.client.get() call.
2. You could store a unique pymemcache client for each thread in your application, perhaps in thread-local storage (see threading.local, for example), and use that instead of storing a single, shared client in self.client.

[jogo]

Closing as the question has been addressed, if you still have questions please re-open