cachex

cachex copied to clipboard

Is this suitable for kubernetes? It seems you have to define the nodes up front, and cant change membership while running? Would be a great features to have for dynamic node memebrship with libcluster.

is this true that cachex wouldn't work well in something like ECS?

I also found this to be a pretty big bummer, seems like it could be something easily remedied. We've implemented our own distributed cache around this tool and was hopping to skip that for a new service and take advantage of this feature.

I dont have the faintest idea if it would work, but perhaps the :nodes options in start_link could be a function that returns the node list. So when fetching/deleting/inserting/etc, the module could call this function and apply it in all returned nodes.

Hi all!

Happy to work on this, given there's traction in the upvotes. I'm not sure exactly what the best option here is, so I'm open to suggestions. I obviously would like to keep the distribution logic fairly straightforward (after all, this is an in-memory caching lib not a distributed data lib).

@mattathompson given you've done a lot of the work for this already, how exactly did you implement it? Is it something that can be shared or is it proprietary?

@gfviegas cool idea, I have no clue how practical it is but sounds fairly elegant to me (at least at a glance).

I'm not scheduling it for a specific version, but v3.7.0 is coming fairly soon and if we can figure this out I'm happy to include it in there! If it's a case of just updating the nodes in the cache, this is simple - i.e. something else is taking care of nodes coming up/down and just updating the reference in Cachex.

Note: I've since migrated my clusterized app using Cachex to Nebulex. It implements several adapters, including a Cachex one.

I'd suggest discussing with Nebulex owners ideas on how clusters could be natively supported on better supported.

I'm fairly confident the way to solve this is to migrate from jump hashing (numeric) to something like libring for key hashing. We can then simply add two new functions to the main Cachex API:

• add_node/1
• remove_node/1

Which will make this transparent, and people can plug into it via whatever clustering lib they decide to use (even if it's just :net_kernel.monitor_nodes/2). This doesn't require much change, except that the internal state of the nodes list will have to move to a Ring. I don't believe this has to result in a major, but perhaps it's best to do so.

Maybe @bitwalker can chime in here as to whether libring seems like a good fit?