ability to exclude specific folders when use --filter generating configs

[rjsigma]

We can use --filter command line parameters to limit configuration generation within a certain glob pattern. However, there is no easy way to exclude some specific folders within the glob pattern. For example, if we have following directory structure

- a
    |- b
    |- c
    |- d

there is no easy way to specify following:

• all subdirectories of a except d should use workflow1
• while d should use workflow2

[gmaghera]

You can do this with what's available today: a/[^d]* would match workflow1 items, all subdirectories of a, except the one which starts with d.

See this example https://go.dev/play/p/iFJsnv6uBAx, which uses the function Match. In our case, Glob() is used, but the docs say it abides by the same pattern as Match().

[gmaghera]

@rjsigma have you tried the globbing pattern I've suggested?

[SerhiiSokolov]

Hi @gmaghera Unfortunately go Glob is very limited. I have a case where need to generate a config for the folders

live-infrastructure/account_123456789012
live-infrastructure/domain/onename/something-01-121234567890
live-infrastructure/anotherdomain/second_name/somethingtwo-01-131234567890

According to the https://www.digitalocean.com/community/tools/glob?comments=true&glob=%2A%2A%2F%2A%2B%28123456789012%7C121234567890%7C131234567890%29&matches=false&tests=%2F%2F%20Imported%20from%20tree%20command&tests=live-infrastructure%2Faccount_123456789012&tests=live-infrastructure%2Fanotherdomain%2Fsecond_name%2Fsomethingtwo-01-131234567890&tests=live-infrastructure%2Fdomain%2Fonename%2Fsomething-01-121234567890 the pattern **/*+(123456789012|121234567890|131234567890) should cover this structure. But If I use provided example from https://go.dev/play/p/iFJsnv6uBAx and try to set my pattern it does not work and seems it happens because go match is very limited according to the documentation

pattern:
	{ term }
term:
	'*'         matches any sequence of non-Separator characters
	'?'         matches any single non-Separator character
	'[' [ '^' ] { character-range } ']'
	            character class (must be non-empty)
	c           matches character c (c != '*', '?', '\\', '[')
	'\\' c      matches character c

character-range:
	c           matches character c (c != '\\', '-', ']')
	'\\' c      matches character c
	lo '-' hi   matches character c for lo <= c <= hi

I wonder if you would be able to figure out how to solve my problem Thanks

[azend]

I also need a feature to be able to exclude certain files/directories from a repo that causes errors and no resulting yaml output. Since there's no way to exclude specific paths, filter must be used to include everything but my particular exclusions. However, because the repository is of unknown depth and Go does not support **, this is not possible.

[azend]

I've solved this temporarily by running this tool within a shell script that sets a consistent root path then iterates through all of the non-excluded directories with a --filter for just that directory. Not ideal but it seems to work. When doing this, remember to --preserve-projects and --preserve-workflows as needed or multiple executions will overwrite previous ones.

[rjsigma]

As mentioned by someone above, go glob is not good at excluding pattern.