problem: current service matching algorithm doesn't fits our need and we cannot customize it

[somdoron]

we are using consisting hashing for some services and exact matching for others, current matching algorithm only match by using regular expression. We want to match by a range.

suggested solution: make the service plugable or make the service matching algorithm plugable.

[hintjens]

Could you back up and explain the problem? (You're explaining the solution to an unstated problem here.) What problem does consistent hashing solve?

[somdoron]

OK, will try again. It is actually of series of problems that lead to have consistent hashing:

problem: we cannot do order execution fast enough solution: do it all in memory and without any locks (one thread)

problem: our one thread get busy by the load of orders which cause high latency for traders solution: have multiple execution workers

problem 1: account requests can be handled by different workers which leads to inconsistency (trader able to open position without enough money) problem 2: execution worker is stateful (because stuff like take profit which is an event raised from inside the worker) which can lead to take profit trigger twice solution: make each worker responsible for a range of accounts exclusively

problem: broker doesn't know to which worker to route the request? solution: each worker will tell the broker which range of accounts it is responsible for, when request arrive broker will check to which range the account it belong and will route the request to the correct worker

problem: account id is not an integer, so using range on something that is not an integer is not possible solution: have hashing method (input is account id) that is known to the broker and the workers, worker will register to range of hashes, broker will calculate the hash for each request and will route the request to the correct broker.

[hintjens]

OK, so it's sufficient that a client can route multiple requests back to the same worker instance, right? Then we can balance clients across workers without any special configuration.

Here's the simplest solution. Note that a worker can offer multiple services. So let's give each worker a generated UUID. When it registers to offer the order placement service, it also registers to offer its generated UUID as service. Now, in all replies sent back to the client (via its mailbox), it adds its UUID as metadata. (We don't have a metadata hash field in messages but we could add that).

Now clients can send further requests to the UUID-named service.

Would that work?

[somdoron]

I don't think so, few problems:

• Because we also need to handle stuff like margin call and pending orders accounts are loaded to memory when the worker starts and handling starts even before client send any request.
• Client can connect and disconnect and in your solution it always have to remember the worker UUID.
• Adding a new worker is complicated because client will always send the request to the wrong worker id.

The simplest solution it to service to actually register to the entire range (if the range is 0-1000 it will register 1000 times).

Another solution can be to have a worker manager, before sending a request client will ask the worker manager (throw a service request) what is the correct worker for the user.

The first solution is to naive and register 1000 times for reach worker can be a lot (usually in consistent hashing a 160 bits hash is used).

The second one is too complicated, I have to create another service and any request will take double the time (first to get the worker and then send the request), caching the worker id is a problem if I want to add another service.

[hintjens]

OK. I believe the simplest solution then is a separate service that maps client account numbers into work service names. Clients can call this once at startup. It does not need any specific support in the Malamute broker. You could run such a service as a thread in the same process as the broker.

On Tue, Mar 3, 2015 at 10:49 AM, Doron Somech [email protected] wrote:

I don't think so, few problems:

• Because we also need to handle stuff like margin call and pending orders accounts are loaded to memory when the worker starts and handling starts even before client send any request.
• Client can connect and disconnect and in your solution it always have to remember the worker UUID.
• Adding a new worker is complicated because client will always send the request to the wrong worker id.

The simplest solution it to service to actually register to the entire range (if the range is 0-1000 it will register 1000 times).

Another solution can be to have a worker manager, before sending a request client will ask the worker manager (throw a service request) what is the correct worker for the user.

The first solution is to naive and register 1000 times for reach worker can be a lot (usually in consistent hashing a 160 bits hash is used).

The second one is too complicated, I have to create another service and any request will take double the time (first to get the worker and then send the request), caching the worker id is a problem if I want to add another service.

— Reply to this email directly or view it on GitHub https://github.com/zeromq/malamute/issues/66#issuecomment-76915904.

[leverate-somech]

Only issue with solution is that it will limit us in case of a adding a new service.

I think of two solutions to this problem:

• We will have a reroute logic inside worker to route the request to the correct service.
• in case of a request that does not belong to the worker it will response with an error "wrong worker" which the client will response with re-requesting the worker name.

I prefer the second.

This doesn't solve my problem completely, for web application I actually have web service in front that forward the requests from clients to a broker (that maybe in the future will be replaced with malamute) and the broker send it to the correct service according to the account id. I can see two solutions:

• For each request first ask for the correct worker name
• In webservice have a cache of account id to worker name, if worker doesn't exist for account id first ask for the worker, update cache and then make request. If i get a "wrong worker" I need to update the cache as well.

Anyway it can work but it will be complicated to get around it. Writing custom service plugin or matching algorithm will be simpler.

[hintjens]

OK, here's another design and rationale. You have a technical issue (scale) which you are now pushing into the contract with your clients. This is like selling pizza and telling large clients they have to go pick up their order in several different locations.

What you really want to do IMO is hide this entirely. You provide a single service that is infinitely scalable. Clients do not know, nor care, how you do that. In fact you do your own routing to child threads and/or processes, tracking them as you want. An actor model.

On Tue, Mar 3, 2015 at 3:00 PM, leverate-somech [email protected] wrote:

Only issue with solution is that it will limit us in case of a adding a new service.

I think of two solutions to this problem:

• We will have a reroute logic inside worker to route the request to the correct service.
• in case of a request that does not belong to the worker it will response with an error "wrong worker" which the client will response with re-requesting the worker name.

I prefer the second.

This doesn't solve my problem completely, for web application I actually have web service in front that forward the requests from clients to a broker (that maybe in the future will be replaced with malamute) and the broker send it to the correct service according to the account id. I can see two solutions:

• For each request first ask for the correct worker name
• In webservice have a cache of account id to worker name, if worker doesn't exist for account id first ask for the worker, update cache and then make request. If i get a "wrong worker" I need to update the cache as well.

Anyway it can work but it will be complicated to get around it. Writing custom service plugin or matching algorithm will be simpler.

— Reply to this email directly or view it on GitHub https://github.com/zeromq/malamute/issues/66#issuecomment-76949654.

[somdoron]

This is what we have now (client don't know of workers and send the request to a broker) and I what I planned to do with Malamute with custom service actor or matching algorithm, I don't like the service discovery idea and prefer that the broker will handle this completely.

I'm not sure how do you suggest to plug this to Malamute? If I understand correctly you suggest to have a service that will be responsible for the routing to the workers? I guess it will be inproc service with the host process of malamute?

This will work but is still more complicated solution then just having the ability to customize the matching algorithm.

Let me describe the solution as I understand it:

• We have a new "Execution" service (inproc with the malamute process) .
• Worker send a "register" service request to the "Execution" service through the Malamute with the range it can handle.
• Client send a service request to "Execution" service (through Malamute), execution service look for the correct worker and send a service request to the correct worker (again through Malamute).

So it is like writing a broker which is why we have Malamute and I'm losing some the features the Malamute is doing right now.

It is much more code then just having custom matching algorithm.

Sorry for repeating this but I still think the simplest solution is to have an option to customize the matching algorithm like in stream and mailboxes. It is also solve the pizza problem you mention pretty nicely.

[hintjens]

OK, I'm convinced. On Mar 3, 2015 5:15 PM, "Doron Somech" [email protected] wrote:

This is what we have now (client don't know of workers and send the request to a broker) and I what I planned to do with Malamute with custom service actor or matching algorithm, I don't like the service discovery idea and prefer that the broker will handle this completely.

I'm not sure how do you suggest to plug this to Malamute? If I understand correctly you suggest to have a service that will be responsible for the routing to the workers? I guess it will be inproc service with the host process of malamute?

This will work but is still more complicated solution then just having the ability to customize the matching algorithm.

Let me describe the solution as I understand it:

• We have a new "Execution" service (inproc with the malamute process) .
• Worker send a "register" service request to the "Execution" service through the Malamute with the range it can handle.
• Client send a service request to "Execution" service (through Malamute), execution service look for the correct worker and send a service request to the correct worker (again through Malamute).

So it is like writing a broker which is why we have Malamute and I'm losing some the features the Malamute is doing right now.

It is much more code then just having custom matching algorithm.

Sorry for repeating this but I still think the simplest solution is to have an option to customize the matching algorithm like in stream and mailboxes. It is also solve the pizza problem you mention pretty nicely.

— Reply to this email directly or view it on GitHub https://github.com/zeromq/malamute/issues/66#issuecomment-76976697.

[somdoron]

thanks