yq icon indicating copy to clipboard operation
yq copied to clipboard

Multiplication of nodes with identical names but different tags merges values

Open Nyefan opened this issue 2 years ago • 3 comments

Describe the bug A clear and concise description of what the bug is.

Note that any how to questions should be posted in the discussion board and not raised as an issue.

Version of yq: 4.33.3 Operating system: mac Installed via: homebrew

Input Yaml Concise yaml document(s) (as simple as possible to show the bug, please keep it to 10 lines or less) data1.yml:

cart:
  banana: !!fruit
    skin: true
    seeds: false
    isFruit: true

data2.yml:

cart:
  banana: !!berry
    skin: true
    seeds: false
    isFruit: false

Command The command you ran:

yq eval-all 'select(fileIndex == 0) * select(fileIndex == 1)' data1.yml data2.yml

Actual behavior

cart:
  banana: !!berry
    skin: true
    seeds: false
    isFruit: false

Expected behavior

cart:
  banana: !!fruit
    skin: true
    seeds: false
    isFruit: true
  banana: !!berry
    skin: true
    seeds: false
    isFruit: false

Additional context Using:

data1.yml:

cart:
  bananas: !!fruit
    skin: true
    seeds: false
    isFruit: true

the above command resolves correctly to:

cart:
  bananas: !!fruit
    skin: true
    seeds: false
    isFruit: true
  banana: !!berry
    skin: true
    seeds: false
    isFruit: false

According to the yaml spec, Two nodes must have the same tag and content to be equal., so I believe this to be a bug. The issue is also present in the case of application-specific tags (! rather than !!).

Nyefan avatar Apr 24 '23 17:04 Nyefan

Separately, in the evaluation of application-specific tags, the merge order of the tag itself is backwards, so the result is:

cart:
  banana: !fruit
    skin: true
    seeds: false
    isFruit: false

Nyefan avatar Apr 24 '23 17:04 Nyefan

That right, it overwrites the values. You cannot have two keys with the same value in a map.

Another way to think of it is merging two maps, one:

a: hello

with another

a: 123

First one has a '!!str' tag and the other has a '!!int' tag. The result of the merge is you get

a: 123

That is the value is overwritten.

mikefarah avatar Apr 27 '23 03:04 mikefarah

Ah, that does make sense, thank you. In that case I suppose the only potential bug is the inconsistent behavior between replacement of global and application-specific tags, though I may be missing something there as well:

# yq ea 'select(di == 0)* select(di ==1)' - <<<$(<yaml1) <<<$(<yaml2)
cart:
  banana: !!berry
    skin: true
    seeds: false
    isFruit: false
# yq ea 'select(di == 0)* select(di ==1)' - <<<$(sed 's/!!/!/g' yaml1) <<<$(sed 's/!!/!/g' yaml2)
cart:
  banana: !fruit
    skin: true
    seeds: false
    isFruit: false

Nyefan avatar May 01 '23 14:05 Nyefan