Support "reserved field"

[rm1138]

Hello, I found that when the proto file contains a "reserved field" will cause the encode and decode fucntion fail.

This this the demo project I created.

demo

Thanks

[bitwalker]

@xinz When running tests with ex_unit_clustered_case, you want to disable libcluster (or more precisely, use the Erlang strategy, which just uses normal Erlang distribution). This library handles spinning up a cluster and connecting the nodes for you; you just have to design tests which put your application through the kinds of failure modes you wish to test (i.e. partitions, crashes).

Could you clarify what you meant by doesn't work in this scenario for me?

[xinz]

@bitwalker Sorry, reply now, after some investigation, I can successfully using ex_unit_clustered_case for my clustering test with swarm and libcluster, for my previous fail case that I was using phoenix_pubsub (version 1.1) as well, I find publish broadcast cannot send to all nodes even though they are connected with each other, maybe there are some missing when setupphoenix_pubsub, when broadcast to subscribers Phoenix.PubSub.PG2Server.get_members won't return the expected members in the cluster testing.

Before using ex_unit_clustered_case, I also try copy-paste and tweak the cluster testing from phoenix_pubsub and swarm, and finally I was successful, but there are a lot of duplicated and complex stuff to adapt. :fearful:

As a final solution, I remove phoenix_pubsub and implement simple pub/sub by myself base on swarm, it works as expected, using ex_unit_clustered_case can really force on the cluster testing of the user application.

I have some items need your advice, please kindly help if possible:

1, I find in my cluster testing, I also need manually add the following code for ensure_all_started for swarm, why ensure my_app started is not enough? Anything I'm missing or it's expected?

  node_setup do
     Application.ensure_all_started(:swarm)
     Application.ensure_all_started(:my_app)
  end

2, Cluster.call vs :rpc.call

At first, I think they're the same thing, Cluster.call is just wrapper for :rpc.call for ExUnit.ClusteredCase, but I find an unexpected failed test case that when I want to parallelly send requests to one node, using Cluster.call even in an async Task, they will send request one by one, but it's not my purpose, using :rpc.call in an async Task will work as expected, this fail case takes some time for checking my application logic, but this "one-by-one" behavior is made from testing. :confused:

Do you have any plan or idea to clarify the usage of Cluster.call, maybe some document I'm missing?

Again, thanks for your brilliant tool for opensource. 😎

[xinz]

Hi @bitwalker, I still occur an error for my test case with swarm, here is a reproducible sample repo for reference: https://github.com/xinz/swarm_test

After clone and add deps for this repo in the local, run mix test will see the following error:

[email protected]:
18:31:38.244 [info]  Application ex_unit_clustered_case exited: exited in: ExUnit.ClusteredCase.App.start(:normal, [])
    ** (EXIT) an exception was raised:
        ** (MatchError) no match of right hand side value: {:error, {{:badmatch, {:error, :eaddrinuse}}, [{:erl_boot_server, :init, 1, [file: 'erl_boot_server.erl', line: 188]}, {:gen_server, :init_it, 2, [file: 'gen_server.erl', line: 374]}, {:gen_server, :init_it, 6, [file: 'gen_server.erl', line: 342]}, {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 249]}]}}
            (ex_unit_clustered_case) lib/app.ex:8: ExUnit.ClusteredCase.App.start/2
            (kernel) application_master.erl:277: :application_master.start_it_old/4

[email protected]:
18:31:38.244 [info]  Application ex_unit_clustered_case exited: exited in: ExUnit.ClusteredCase.App.start(:normal, [])
    ** (EXIT) an exception was raised:
        ** (MatchError) no match of right hand side value: {:error, {{:badmatch, {:error, :eaddrinuse}}, [{:erl_boot_server, :init, 1, [file: 'erl_boot_server.erl', line: 188]}, {:gen_server, :init_it, 2, [file: 'gen_server.erl', line: 374]}, {:gen_server, :init_it, 6, [file: 'gen_server.erl', line: 342]}, {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 249]}]}}
            (ex_unit_clustered_case) lib/app.ex:8: ExUnit.ClusteredCase.App.start/2
            (kernel) application_master.erl:277: :application_master.start_it_old/4

Could you please advise anything on it?

[xadhoom]

I'm having the same error, and I suspect that happens because ex_unit_clustered_case is started also on other nodes (apart from the primary) and erl_boot_server fails to start.

here's what happens (on the spawned nodes): {:error, {:ex_unit_clustered_case, {:bad_return, {{ExUnit.ClusteredCase.App, :start, [:normal, []]}, {:EXIT, {{:badmatch, {:error, {{:badmatch, {:error, :eaddrinuse}}, [{:erl_boot_server, :init, 1, [file: 'erl_boot_server.erl', line: 188]}, {:gen_server, :init_it, 2, [file: 'gen_server.erl', line: 374]}, {:gen_server, :init_it, 6, [file: 'gen_server.erl', line: 342]}, {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 249]}]}}}, [{ExUnit.ClusteredCase.App, :start, 2, [file: 'lib/app.ex', line: 8]}, {:application_master, :start_it_old, 4, [file: 'application_master.erl', line: 277]}]}}}}}}

(sorry for the bad formatting, but had to write output on a file, since was not able to log what is happening inside the :node_setup callbacl)

[xadhoom]

What is strange, is that :erl_boot_server should use a random port. But as a proof, disabling :erl_boot_server.start_link in ExUnit.ClusteredCase.App makes it work (obviously this breaks "sending code" to other nodes, but just for test)

[xadhoom]

ok, the error starts from https://github.com/erlang/otp/blob/master/lib/kernel/src/erl_boot_server.erl#L188, basically erl_boot_server listens always to a static udp port, so starting multiple times :ex_unit_clustered_case on the same node will lead to errors.

I think that the resolution is not to start :ex_unit_clustered_case dep in the spawned nodes, but how?