Command Migration: ('JSON.NUMINCRBY', 'JSON.NUMMULTBY', 'JSON.TOGGLE', 'JSON.FORGET')

[AshwinKul28]

This issue tracks the migration of the mentioned commands - ('JSON.NUMINCRBY', 'JSON.NUMMULTBY', 'JSON.TOGGLE', 'JSON.FORGET') to make them compatible across the three protocols supported by the Dice database: RESP, HTTP, and WebSocket. DiceDB now supports more than one protocols (Resp/http/websocket). We don't want eval operation of each command to be strictly bound with any of the protocols. Currently eval function return statements of each command is bounded to the RESP protocol.

The migration is required because:

• The current implementation is specific to the RESP protocol
• Evaluation functions of each command must be refactored to ensure protocol-agnostic behavior.
• Evaluation functions should return raw responses instead of protocol specific responses.

The goal is to make the command logic protocol-independent, allowing all three protocols to call the same core functionality seamlessly.

Requirements

• Refactor the evaluation function of [Command Name] to be generic.
• Ensure the evaluation function does not include protocol-specific logic.

Migration Steps

1. Analyze Current Implementation

   • Review the current codebase to understand how the command logic is implemented.

2. Refactor Return Logic

   • Create a new function with the function definition as evalXXX(args []string, store *dstore.Store) *EvalResponse under the file /internal/eval/store_eval.go
   • Analyse return statements of the eval and modify them to send raw types without encoding
   • Use errors in the return statements from /errors/migrated_errors.go file
   • Use perdefined responses from /internal/clientio/resp.go file as RespType variables

3. Command/Worker specific Changes

   • Make IsMigrated flag to true in the commands information under /internal/eval/commands.go file.
   • Use the newly written eval function against the NewEval parameter in the same command structure.
   • Delete the old eval function from the /internal/eval/eval.go file.
   • Add the migrated command to the /internal/worker/CommandsMeta map and make it's type as SingleShard

4. Update Unit Tests

   • Refactor existing unit tests to accommodate the new implementation.
   • Add new unit tests if necessary to cover all possible cases.

5. Integration Tests

   • Run all integration tests to ensure successful migration.
   • Ensure that each protocol (RESP, HTTP, WebSocket) works correctly after migration.

Checklist

• [ ] Migrated the evalXXX function with the latest definition
• [ ] Update or add unit tests for the new implementation.
• [ ] All unit tests pass successfully.
• [ ] Ensure all integration tests pass successfully.

Additional Notes

• Describe any edge cases that need to be handled in the generic function.
• Mention any protocol-specific optimizations required in wrappers.

If there are any questions or concerns about this migration, please mention them here.

Related Issues/PRs

• Sample implementation for the Get, Set, GetSet and SetEx can be found in this Pull Request.

[dankot12]

Hi @AshwinKul28 , can I please pick this up? Thank you :)

[AshwinKul28]

Thanks @dankot12 go for it! 🚀

[AshwinKul28]

HI @dankot12 , I hope you are doing well. Do you happen to have any updates on this? If you've any doubts please let us know on the discord.

[vpsinghg]

@dankot12 , could you please confirm if you're currently working on this or something else?

@AshwinKul28 can you assign an issue that hasn't received any updates from the assignee for a long time?

[AshwinKul28]

Hey @vpsinghg go for it! Thanks 🚀

[vpsinghg]

@AshwinKul28 I observed a issue in json.numincrby and json.nummultby output. As we use "github.com/ohler55/ojg/jp" for generating a expression and use its Get method on json object. But this gives result in a sorted order. Which is different from the output received from redis.

What should be the correct behavior ? DiceDB output

127.0.0.1:7379> JSON.SET number $ '{"a": 2, "b": 10, "c": [15, {"d": 20}]}'
OK
127.0.0.1:7379> JSON.NUMINCRBY number $..* 5
"[25,20,null,null,7,15]"

Redis output

127.0.0.1:6379> JSON.SET number $ '{"a": 2, "b": 10, "c": [15, {"d": 20}]}'
OK
127.0.0.1:6379> JSON.NUMINCRBY number $..* 5
"[7,15,null,20,null,25]"
127.0.0.1:6379> 

[iamskp11]

@AshwinKul28 I observed a issue in json.numincrby and json.nummultby output. As we use "github.com/ohler55/ojg/jp" for generating a expression and use its Get method on json object. But this gives result in a sorted order. Which is different from the output received from redis.

What should be the correct behavior ? DiceDB output

127.0.0.1:7379> JSON.SET number $ '{"a": 2, "b": 10, "c": [15, {"d": 20}]}'
OK
127.0.0.1:7379> JSON.NUMINCRBY number $..* 5
"[25,20,null,null,7,15]"

Redis output

127.0.0.1:6379> JSON.SET number $ '{"a": 2, "b": 10, "c": [15, {"d": 20}]}'
OK
127.0.0.1:6379> JSON.NUMINCRBY number $..* 5
"[7,15,null,20,null,25]"
127.0.0.1:6379> 

Hey @vpsinghg , the keys orderings inside JSON are not deterministic. So, I think ordering is not an issue in this case since the given path is matching all keys.

[vpsinghg]

Can any one explain following. Why does response of setupData command is ERR syntax error response in case of websocket as executer?

		{
			name: "Forget nested field",
                       setupData: `JSON.SET user $ {"name":"Tom","address":{"city":"New York","zip":"10001"}}`,
			commands: []string{
				"JSON.FORGET user $.address.city",
				"JSON.GET user $",
			},
			expected: []interface{}{"OK", float64(1), `{"name":"Tom","address":{"zip":"10001"}}`},
			cleanUp:  []string{"DEL user"},
		},

I've noticed that if I pass a string with a space (e.g., "name":"New York"), the command parsing and argument splitting in the command parser does not handle this scenario correctly. Currently observed this with JSON.SET command

[AshwinKul28]

@apoorvyadav1111 @lucifercr07 Can you please take care of this websocket Question? Thanks

[lucifercr07]

Can any one explain following. Why does response of setupData command is ERR syntax error response in case of websocket as executer?

		{
			name: "Forget nested field",
                       setupData: `JSON.SET user $ {"name":"Tom","address":{"city":"New York","zip":"10001"}}`,
			commands: []string{
				"JSON.FORGET user $.address.city",
				"JSON.GET user $",
			},
			expected: []interface{}{"OK", float64(1), `{"name":"Tom","address":{"zip":"10001"}}`},
			cleanUp:  []string{"DEL user"},
		},

I've noticed that if I pass a string with a space (e.g., "name":"New York"), the command parsing and argument splitting in the command parser does not handle this scenario correctly. Currently observed this with JSON.SET command

I believe there's an issue with websocket parser, for now can you create without spaces the test cases, so that we're not blocked on this. Will create an issue to get this fixed, shouldn't be a blocker for migration. You can try below as message:

JSON.SET user $ {"name":"Tom","address":{"city":"NewYork","zip":"10001"}}