deep_copy_list working incorrectly

[md2k]

trafficstars

Describe the Bug

Option deep_copy_list working incorrectly with utils_deep_merge_yaml (not checked utils_deep_merge_json , but probably same).

deep merge configured with : append_list = false , deep_copy_list = true Input of 2 yaml:

---
test:
  my_list:
     - item1
     - item2

---
test:
  my_list:
     - item1
     - item3
     - item4

Getting as result:

---
test:
  my_list:
     - item1
     - item2

so 2nd Yaml list ignored. In version 1.23 if append_list and deep_copy_list both enabled, result as above, in versions below 1.23 result will be as expected, but probably because append_list in actions.

Expected Behavior

---
test:
  my_list:
     - item1
     - item2
     - item3
     - item4

Steps to Reproduce

Merge any yaml (even from example) with utils_deep_merge_yaml and deep_copy_list enabled

Screenshots

No response

Environment

OS [Mac M3, Mac Intel, Linux], module version 1.23, 1.22, 1.21, 1.20, 1.19, 1.18

Additional Context

Need to merge complex yaml files, with lists in them, in multiple cases it is required to have merge of lists and ensure we not have duplicates in final outputs, so for that we trying to use deep_copy_list , but seems doesn't work.

[pyrxm]

Can confirm we're also experiencing this, it does appear to have been a change in 1.23 that has caused this.

https://github.com/cloudposse/terraform-provider-utils/pull/400/files

Looks like atmos updated and maybe the merge package has changed significantly between 1.70 (provider v1.22) and 1.79 (provider v1.23)... maybe this release? https://github.com/cloudposse/atmos/releases/tag/v1.76.0

Just my 2¢.

[md2k]

Is any suggestion from maintainers, @aknysh ?

[md2k]

Guys, @aknysh , @goruha , @osterman you have complete regression in provider after version 1.22, append list and deep merge doesn't work anymore. Is this project considered as absolute and not going to be maintained and or accept new PRs ?

[md2k]

I found the issue and it is not related to cloudposse provider and Atmos, it is issue within mergo. But my question about contribution and accepting PRs are still valid.

[jgournet]

@md2k : sorry for asking you, but since maintainers are not answering: have you found a workaround for this issue ?

(I tried several versions now, and the least worst I could get is with 1.21.0 and 1.22.0:

Changes to Outputs:
  + deep_merge_output = {
      + test = {
          + my_list = [
              + "item1",
              + "item2",
              + "item1",
              + "item3",
              + "item4",
            ]
        }
    }

not great, but close enough

[md2k]

@jgournet, this is problem of upstream library mergo, and author if it seems wont fix or not rly active on current version. the only way i can see is to fork mergo, fork provider (or write new one) where will be used forked fixed mergo, but i not sure yet that i want to go that route.