Incorrect Warning: ls * | grep -v something [SC2010]

[7Pawns]

For bugs

• SC2010
• 0.9.0
• [] The rule's wiki page does not already cover this (e.g. https://shellcheck.net/wiki/SC2086) Says there may be exceptions but still pops an error.
• [x] I tried on https://www.shellcheck.net/ and verified that this is still a problem on the latest commit

Here's a snippet or screenshot that shows the problem:

#!/bin/bash
ls * | grep -v something

Here's what shellcheck currently says:

-- SC2010 (warning): Don't use ls | grep. Use a glob or a for loop with a condition to allow non-alphanumeric filenames.

Here's what I wanted or expected to see:

nothing, there is no better way to do this

[brother]

Isn't that the same as grep -v something ./*?

[7Pawns]

@brother it isn't, grep will list all files regardless of the exclusion

[brother]

grep the list of files not inside the files. ofc. thanks =)

[7Pawns]

@brother what do you mean?

[brother]

that I misread the initial problem. got it now.

[brother]

The solution to have a file list where some string is exclude would be along the lines of ./*!(something). Having a ignore directive here is probably good enough.

[superbonaci]

This is the message I get currently:

In script line 23:
ls * | grep -v '*.pdf'
^-- SC2010 (warning): Don't use ls | grep. Use a glob or a for loop with a condition to allow non-alphanumeric filenames.
   ^-- SC2035 (info): Use ./*glob* or -- *glob* so names with dashes won't become options.

The biggest issue I see is, ls expands the first level of directories, for which there is no for equivalent. Most times people really don't intent to expand directories when using ls (they don't know), so this is what is wrong in the first place, which is not mentioned:

So

ls * | grep -v something

Is not the same as

for i in * ; do echo $i ; done | grep -v something

Also, it's not true that ls does not allow non-alphanumeric filenames, they outputs it perfectly, however if you don't use additional flags, filenames are mixed in the same line:

ls *
test:
aa$%	test2

ls test/*
test/aa$%

test/test2:
aa %$&bb.txt 	file.txt

Sometimes is also difficult to know if the file has a space at the end, while at the beginning is easier to notice. Different ls versions display it differently:

ls -A1
aa %$&bb.txt
file.txt

gls -A1
'aa %$&bb.txt '
file.txt

Maybe the message could be formatted like this:

In script line 23:
ls * | grep -v '*.pdf'
^-- SC2010 (warning): Don't use ls | grep if you don't intend to expand first-level directories automatically.
Use "-1" or "-l" flags to separate by line so non-alphanumeric filenames are recognised easier.
Use a glob or a for loop with a condition to work with non-alphanumeric filenames as arguments for another program.
   ^-- SC2035 (info): Use ./*glob* or -- *glob* so names with dashes won't become options.

[7Pawns]

I think the line of code I supplied should be ok in the first place, I am not sure more warnings will be good, and it is a pretty broad one aswell

[deryni]

The ls | grep code is absolutely not going to work correctly for filenames with newlines in them at minimum.

A glob absolutely works if you need it to expand down into a single directory you just need to use */* as a glob in addition to *.

That said, for almost any usage I can imagine here I'd expect that using find is probably the best solution (even if that just means piping find into a while loop or reading it into an array).

[mohd-akram]

Using a glob with ls is very bad. If you do ls *, you will print the contents of subdirectories, and the output will look something like this if there are multiple directories:

$ ls *
directory:
file1	file2

directory2:
file3	file4

One alternative is printf '%s\n' *.

This doesn't just apply for globs, it applies in any scenario where it is unknown whether the argument will expand to a file or directory, eg: ls .git -- well, .git can be a file (eg. inside git submodules) so ls would just print .git again.

The newline concern is unimportant and overblown IMO -- every UNIX tool is newline-based, and if you have newlines in your filenames, you have bigger problems. This is the same with filenames starting with -. While one can try to use -- everywhere, it is easy to miss and almost futile, but at least it's far more likely that a file begin with - than have a newline in it.