Support for AWS ElastiCache "config" command (needed to support cluster configuration)

[raff]

The AWS implementation of memcached adds a new command "config" (for now only "config get cluster") that returns the list of hosts in the cluster, for auto-discovery. I would like to be able to use pymemcache on AWS and get the cluster configuration:

http://docs.aws.amazon.com/AmazonElastiCache/latest/UserGuide/AutoDiscovery.AddingToYourClientLibrary.html

A couple of ways:

• a low level command (like "_misc_cmd") that can return multiple lines for the result (_misc_cmd currently reads just one line). The "config" command currently returns 2 lines terminated with "END")
• update _fetch_cmd to add support for CONFIG (but if AWS adds more subcommands, the format may change).

I would be fine with the first way (and then expose either a "config" or "aws_config" command or add a generic high-level command that would allow to send more "non-standard" requests. I have hacked something together that implement this (with a new "aws_config" method on the base client class) and I can submit a pull request.

[jogo]

Hi @raff, if you're still interested a pull request would be welcome.

[lfdesousa]

Something like this? https://github.com/lfdesousa/elasticache-auto-discovery/tree/python3.6

[martinnj]

So I've been walking around thinking a bit about this, wanting to give it a shot.

Is the target to:

• Create a whole new client?
• Create a new client that inherits from HashClient?
• Add as a feature to the HashClient?
• Create a tool that can return a server list, like @lfdesousa posted above? (list(tuple(server, port))
• Create a tool that can create a brand new HashClient?

The most user-friendly way, in my opinion, would be a new client that reconfigures itself as the config changes, but a wholly new client is also a lot of new code to write and test.

Is there any preferences from your side @jogo or should I just throw something together that works for me. My first thoughts would be to inherit from the HashClient and see if I can make that work?

[jogo]

Inheriting from hashclient sounds like a good idea to me, @jparise thoughts? Thanks for working on this.

[takeda]

I see that #395 was added are there still plans to integrate the auto discovery?

[martinnj]

I see that #395 was added are there still plans to integrate the auto discovery?

There was a lot of intentions on my part. But I got buried in work when I was trying to figure out a good semantic for "This might be a connection issue so retry the host in a moment" vs "The config has changed, so let's refresh the server list" and planning tests.

I don't think the maintainers ever had any plans for adding it themselves since they don't use ElastiCache. (or didn't when I last looked at it.)

[jogo]

I don't think the maintainers ever had any plans for adding it themselves since they don't use ElastiCache. (or didn't when I last looked at it.)

That is still correct, we are more than happy to accept PRs but have no plans of adding it ourselves.