etcdctl: fix json output

[kstrifonoff]

Fix #19262

Example output of etcdctl member add test-node --learner=true --peer-urls=http://127.0.0.1:22380 --write-out json --hex command:

Before:

{
  "header": {
    "cluster_id": 14841639068965178418,
    "member_id": 10276657743932975437,
    "raft_term": 2
  },
  "member": {
    "ID": 17436648322565278587,
    "peerURLs": ["http://127.0.0.1:22380"],
    "isLearner": true
  },
  "members": [
    {
      "ID": 10276657743932975437,
      "name": "default",
      "peerURLs": ["http://localhost:2380"],
      "clientURLs": ["http://localhost:2379"]
    },
    {
      "ID": 17436648322565278587,
      "peerURLs": ["http://127.0.0.1:22380"],
      "isLearner": true
    }
  ]
}

After

{
  "header": {
    "cluster_id": "cdf818194e3a8c32",
    "member_id": "8e9e05c52164694d",
    "raft_term": 2
  },
  "member": {
    "ID": "5bdeb7bdb679126b",
    "peerURLs": ["http://127.0.0.1:22380"],
    "isLearner": true
  },
  "members": [
    {
      "ID": "5bdeb7bdb679126b",
      "peerURLs": ["http://127.0.0.1:22380"],
      "isLearner": true
    },
    {
      "ID": "8e9e05c52164694d",
      "name": "default",
      "peerURLs": ["http://localhost:2380"],
      "clientURLs": ["http://localhost:2379"]
    }
  ]
}

[k8s-ci-robot]

Hi @kstrifonoff. Thanks for your PR.

I'm waiting for a etcd-io member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[kstrifonoff]

/cc @ivanvc

[ivanvc]

Hi @kstrifonoff, thanks for the pull request. I'm respectful to the original contributor who asked to work on this. Our guideline is to re-assign after a week of no answer when asking a contributor for updates.

A couple of suggestions by quickly looking at the code.

1. The imports contain a lint error. Running make verify-lint locally will show you the error.
2. The code doesn't have unit tests for this, but there's an e2e test, TestCtlV3MemberListWithHex. I think the test should pass, but adding your new cases would be good.
3. I suggest reviewing the code from printKV. I think it would be better to follow that implementation approach, i.e., there would be less repetition with the if conditions.

Thanks again for your pull request :)

[kstrifonoff]

Hi @ivanvc ,

I’ve made some changes to my initial suggestions based on your feedback.

Now, the “JSON with Hex” implementation is just a slight extension of the default “printJSON” implementation. I think it should probably be moved into the “utils” package.

Here are a few things I noticed:

1. The implementation assumes marshalling back and forth the default JSON string, which can be inefficient, but I assume it’s still okay for the CLI.
2. Also, the current approach makes it easy to add “json with hex” output for all the rest of commands that support JSON output.
3. I’m not sure what kind of end-to-end test should be done here. What should it cover? It’s also not clear to me what the current end-to-end test actually checks.

[ivanvc]

/ok-to-test

[ivanvc]

I’m not sure what kind of end-to-end test should be done here. What should it cover? It’s also not clear to me what the current end-to-end test actually checks.

The e2e tests already exist, my suggestion is to expand them to cover the new cases you added: https://github.com/etcd-io/etcd/blob/9f1709e015640838269bfbecae9ee2be41b47939/tests/e2e/ctl_v3_member_test.go#L206-L255

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 95.74468% with 2 lines in your changes missing coverage. Please review.

Project coverage is 69.22%. Comparing base (d744c19) to head (b42b5f9). Report is 11 commits behind head on main.

Files with missing lines	Patch %	Lines
etcdctl/ctlv3/command/printer_json.go	95.74%	2 Missing :warning:

Additional details and impacted files

Files with missing lines	Coverage Δ
etcdctl/ctlv3/command/printer_json.go	46.60% <95.74%> (+46.60%)	:arrow_up:

... and 20 files with indirect coverage changes

@@            Coverage Diff             @@
##             main   #19469      +/-   ##
==========================================
- Coverage   69.28%   69.22%   -0.06%     
==========================================
  Files         414      414              
  Lines       34408    34455      +47     
==========================================
+ Hits        23840    23853      +13     
- Misses       9180     9203      +23     
- Partials     1388     1399      +11     

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update d744c19...b42b5f9. Read the comment docs.

:rocket: New features to boost your workflow:

• :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[kstrifonoff]

hi @ivanvc Could you give it another look? I’ve made a few changes since we last spoke. The initial solution had a bug, so I’ve fixed it.

I’ve also fixed all the remaining member and existing endpoint commands. Now, you can also use them with json and hex output. I’ve also extended the e2e tests as suggested.

Let me know if you have any questions!

[kstrifonoff]

@ivanvc just a friendly reminder: could you please review this once you have some time?

[kstrifonoff]

/cc @fuweid

Could you please review this PR?

[kstrifonoff]

/retest

[kstrifonoff]

/retest

[fuweid]

cc @ivanvc @ahrtr @serathius

[ahrtr]

thx for the contribution. But it seems that you introduced an inconsistent way as one existing implementation,

https://github.com/etcd-io/etcd/blob/b3b9e986e037727a53ea08e5d7f8672cd5397fc7/etcdctl/ctlv3/command/printer_json.go#L43-L60

Also this is a generic problem, other commands have similar problem, but you only fixed MemberAddResponse.

It'd be good that we have a overall plan on how to resolve the json output problem for all commands in a consistent way.

[kstrifonoff]

@ahrtr initially, I modified all the other commands in this way. However, it was suggested to minimize changes to address the original issue and move refactoring out of the scope of this PR.

You can take a look at the proposed overall solution here: https://github.com/kstrifonoff/etcd/blob/refactor/hex-json-printer/etcdctl/ctlv3/command/printer_json.go

I agree that it should align with the rest of the commands. However, I’m not sure how to provide it in a better way (either within one PR or multiple PRs).

Could you please let me know your ideas?

[ahrtr]

thx for the clarification.

Breaking down changes into smaller PRs is good. Please clarify your plan in https://github.com/etcd-io/etcd/issues/19262 ( I just reopened it) and get approval first (please bring it to our community meeting. Note: I am supportive on it). Afterwards, then resolve other commands one by one per the plan.

[ahrtr]

Another comment is that there is no test. Could you also add a unit test for each command you updated?

[ahrtr]

Another comment is that there is no test. Could you also add a unit test for each command you updated?

Also please provide example outputs before and after the change for reference/comparison in each PR (including this one).

[ahrtr]

Overall looks good with a comment (adding unit test) https://github.com/etcd-io/etcd/pull/19469#discussion_r2144585903

Please also list all the commands which we need to make similar change in the issue, so that we don't miss any one of them.

[ahrtr]

pls rebase this PR instead of merging main into your dev branch. Also pls consider adding an unit test as mentioned in https://github.com/etcd-io/etcd/pull/19469#discussion_r2144585903

[kstrifonoff]

pls rebase this PR instead of merging main into your dev branch.

@ahrtr Thanks for the comment. In any case, I intend to squash all the commits in this branch once I complete the tasks before submitting the PR for re-approval. In such a scenario, I believe it doesn’t matter whether I rebase or merge.

However, I believe it might be worthwhile to convert this to a draft until then.

[kstrifonoff]

/retest

[kstrifonoff]

/retest

[k8s-ci-robot]

@kstrifonoff: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci-etcd-robustness-release36-amd64	311626f8043fddee17dfb27ddd4a5a6b09d49fe9	link	true	/test ci-etcd-robustness-release36-amd64

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[kstrifonoff]

/retest

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ahrtr, fuweid, kstrifonoff

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• ~~OWNERS~~ [ahrtr,fuweid]

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

[Elbehery]

@ivanvc did @kstrifonoff address your review ?