Option `--front-matter` is only applied to first file, not subsequent

[Boerseth]

Version of yq: v4.34.2 Operating system: Ubuntu Linux Installed via: Don't remember, probably snap

Input Yaml data1.yml:

---
b: banana

data2.md:

---
b: botato
---
not valid
yaml: code

Command

yq eval '.' data1.yml
# OK
yq eval-all '.' data1.yml data1.yml
# OK
yq --front-matter extract eval '.' data2.md
# OK
yq --front-matter extract eval-all '.' data2.md data2.md
# Error!

Actual behavior

yq --front-matter extract eval-all '.' data2.md data2.md
# Error: bad file 'data2.md': yaml: line 5: mapping values are not allowed in this context

Expected behavior

yq --front-matter extract eval-all '.' data2.md data2.md
# ---
# b: botato
# ---
# b: botato

Additional context I would like a simple way to collect the metadata from a bunch of Markdown files into one array of objects. I've been extracting them into individual temporary yml files using yq and joining them afterwards, but now I want to skip the middle step and collect it all directly using eval-all.

After some trial and a lot of error, it seems as if it's always the second file in the arguments where parsing fails, as if yq doesn't know it's only concerned with the --front-matter.

Another workaround for me could be, since the fields happen to always be the same in my case, something like

head -q -n 5 location/of/files/*.md | yq ea '. as $i ireduce ([]; . + $i)'

but that feels rather hacky. I would much rather that yq handle the metadata from start to finish.

[Boerseth]

I realize my example input doesn't highlight this, so here is further info:

cat <<EOF > first.md
---
c: cat
---
not valid
yaml: 1234
EOF

cat <<EOF > second.md
---
c: cow
---
also not
valid: yaml
EOF

yq ea '.' -f extract first.md second.md
#Error: bad file 'second.md': yaml: line 5: mapping values are not allowed in this context

yq e '.' -f extract first.md second.md
#---
#c: cat
#Error: bad file 'second.md': yaml: line 4: mapping values are not allowed in this context

As seen above, it is the file second.md which causes problems, even though it is not much different from first.md. In the yq e example, you see that first.md is handled fine and outputs before second.md results in an error.

This and other things make it seem likely that it's just that -f is not applied to the second file.

I really wish I knew Go at all, because a bug like this seems like a great opportunity to get into a new repo and do the fix myself. Sad to say I didn't have much luck finding it out reading through the code.

[ngirard]

I would like a simple way to collect the metadata from a bunch of Markdown files into one array of objects.

Hey @Boerseth ,

meanwhile, have you found an alternative way of achieving this ? I'd be interested !

[mikefarah]

yq really only operates at one file at a time, the other file parameters are for merging data in if required.

Best way I can think of doing what you want is something like this:

find *.md -exec yq -f extract {} \; | yq ea '[.]'

Explanation:

1. Use find to list all the md files and pipe each one through to 'yq` to extract the yaml content
2. Pipe that data through yq again, using ea to read all the documents at once and [.] to put them into a single array.