Trailing space in string block transforms to quoted form

[Ri0n]

Describe the bug I use next expression to update YAML item

yq_linux_amd64 -i '
with(.path.to.item.content ; . = strenv(CONTENTS) | . style="literal")' \
myinputfile.yaml

Where CONTENTS keeps some long bash script. It worked well for me until some recent changes the the bash script.

So if before something went wrong IIRC the content in YAML looked like

   |-
      line1
      line2

Then now it looks like

line1\nline2

basically making it unreadable in the YAML

Version of yq: 4.26.1 Operating system: linux Installed via: binary release

Input Yaml Unfortunately the YAML source and the scrips I use are commercial, so I can't post them here and I haven't tried to find the relevant change myself in hope it's something known.

So basically I want the behavior I had initially.

[mikefarah]

I appreciate the script is commercial - they often are. There has been no changes that I'm aware of that should produce that effect. Just tried;

CONTENTS="cat
dog" ./yq -n '.x = strenv(CONTENTS)'

and it resulted:

x: |-
  cat
  dog

Unless you give me a representative (non-commercial :) ) example that I can reproduce the problem with - there's little I can do to help. I've got this guide here that may help as well: https://mikefarah.gitbook.io/yq/operators/string-operators#string-blocks-bash-and-newlines

[Ri0n]

Hi Mike,

I figured it out! One of lines in my bash script had a trailing space. For some reason this make yq ignore the style. So I removed this single trailing space and everything works properly now.

With your example is like putting a space right after cat or dog.

To be honest even if it's a correct behavior it's completely unexpected. And there were no warnings or anything giving a clue the spaces are forbidden with this style.

[mikefarah]

Ooh I haven't seen that before - that is odd.

A trailing space in a block changes to a quoted string - this even happens when opening a file in that format. yq isn't doing anything to the strings itself, this is an issue with the underlying yaml parse go-yaml :/

[mikefarah]

FYI this depends on a fix for https://github.com/go-yaml/yaml/issues/880

[ericdstein]

This issue also occurs with CRLF line endings.

teststring=$(cat <<-END
This is line one.
This is line two.
This is line three.
END
)

dos="$(printf "%s" "$teststring" | unix2dos)"
TEST="$dos" yq -n '.test = strenv(TEST)'

unix="$(printf "%s" "$teststring" | dos2unix)"
TEST="$unix" yq -n '.test = strenv(TEST)'

outputs

test: "This is line one.\r\nThis is line two.\r\nThis is line three."
test: |-
  This is line one.
  This is line two.
  This is line three.

[mikefarah]

FYI - you can use yq to remove trailing spaces in a string block like so:

yq '.myStringBlock |= sub(" *\n", "\n")' file.yaml

[bryant-ferguson]

Duplicate of https://github.com/mikefarah/yq/issues/566