Inlcude and excludes are not handled correctly. [`p in incs and not in excs`]

[gabyx]

What steps did you take:

I have the following:

apiVersion: vendir.k14s.io/v1alpha1
kind: Config

 - path: components/custodian/src
    contents:
      - path: .
        git:
          url: https://gitlab.com/data-custodian/custodian.git
          ref: main
          depth: 1

        includePaths:
          - tools/deploy/manifests/src/**/*
        excludePaths:
          - tools/deploy/manifests/src/deployments

        newRootPath: "tools/deploy/manifests/src"

which should exclude deployments directory but it does not.

What happened: tools/deploy/manifests/src/deployments is vendored even though its excluded in the config.

What did you expect: tools/deploy/manifests/src/deployments is ignored and should not exist.

Anything else you would like to add: Vendir is a nice tool, but its a bit upsetting when basic things as include and excludes are not working as they should: the only correct semantics for this IMO is:

• p is vendored if it matches any includePaths (should be named includeGlobs...) and not any of excludePaths.

#exclude paths are "placed" on top of include paths (optional)

this is the behavior I see that first the path gets excluded and then included again. I think this is just wrong and limits the power of vendir.

Environment:

• vendir version (execute vendir --version): 0.44.0
• OS (e.g. from /etc/os-release): nixos

---

Vote on this request

This is an invitation to the community to vote on issues, to help us prioritize our backlog. Use the "smiley face" up to the right of this comment to vote.

👍 "I would like to see this addressed as soon as possible" 👎 "There are other more important things to focus on right now"

We are also happy to receive and review Pull Requests if you want to help working on this issue.

[joaopapereira]

I was trying to reproduce your problem with the lock file you provided but not sure I can, when I used the lock file above this is what I got

❯ tree .
.
├── components
│   └── custodian
│       └── src
│           ├── helm
│           │   └── mongodb
│           │       ├── Chart.lock
│           │       ├── Chart.yaml
│           │       └── ytt
│           │           ├── _ytt_lib
│           │           │   └── common
│           │           │       └── common.star
│           │           └── values.yaml
│           └── ytt
│               ├── _ytt_lib
│               │   └── common
│               │       └── common.star
│               ├── contract-manager
│               │   ├── _ytt_lib
│               │   │   └── common
│               │   │       └── common.star
│               │   ├── configmap.yaml
│               │   ├── deployment.yaml
│               │   ├── secrets.yaml
│               │   └── service.yaml
│               ├── mongodb
│               │   ├── _ytt_lib
│               │   │   └── common
│               │   │       └── common.star
│               │   └── secrets.yaml
│               ├── openapi
│               │   ├── _ytt_lib
│               │   │   └── common
│               │   │       └── common.star
│               │   ├── deployment.yaml
│               │   ├── ingress.yaml
│               │   └── service.yaml
│               └── schema.yaml
├── vendir.lock.yml
└── vendir.yml

Did you make any changes to the repository?

[gabyx]

sorry yes: I send you the commit sha which produces the error:

https://gitlab.com/data-custodian/custodian/-/commit/39c6718189f79557d0310a419e7e2facb1be8d04

thanks for reproducing this

[joaopapereira]

I updated my vendir.yml file to:

apiVersion: vendir.k14s.io/v1alpha1
kind: Config
directories:
 - path: components/custodian/src
   contents:
      - path: .
        git:
          url: https://gitlab.com/data-custodian/custodian.git
          ref: 39c6718189f79557d0310a419e7e2facb1be8d04
          #depth: 1

        includePaths:
          - tools/deploy/manifests/src/**/*
        excludePaths:
          - tools/deploy/manifests/src/deployments

        newRootPath: "tools/deploy/manifests/src"

And my tree still is:

❯ tree .
.
├── components
│   └── custodian
│       └── src
│           ├── helm
│           │   └── mongodb
│           │       ├── Chart.lock
│           │       ├── Chart.yaml
│           │       └── ytt
│           │           ├── _ytt_lib
│           │           │   └── common
│           │           │       └── common.star
│           │           └── values.yaml
│           └── ytt
│               ├── _ytt_lib
│               │   └── common
│               │       └── common.star
│               ├── contract-manager
│               │   ├── _ytt_lib
│               │   │   └── common
│               │   │       └── common.star
│               │   ├── configmap.yaml
│               │   ├── deployment.yaml
│               │   ├── secrets.yaml
│               │   └── service.yaml
│               ├── mongodb
│               │   ├── _ytt_lib
│               │   │   └── common
│               │   │       └── common.star
│               │   └── secrets.yaml
│               ├── openapi
│               │   ├── _ytt_lib
│               │   │   └── common
│               │   │       └── common.star
│               │   ├── deployment.yaml
│               │   ├── ingress.yaml
│               │   └── service.yaml
│               └── schema.yaml
├── vendir.lock.yml
└── vendir.yml

[gabyx]

Ok, thats weird I need to investigate again when that happened.

[gabyx]

@joaopapereira : The following reproduces the error:

apiVersion: vendir.k14s.io/v1alpha1
kind: Config
directories:
  - path: t
    contents:
      - path: .
        git:
          url: https://gitlab.com/data-custodian/custodian.git
          ref: 85116b6dcf1e9224f2728c879451502fb5317b9f
          #depth: 1

        includePaths:
          - tools/deploy/manifests/src/**/*
        excludePaths:
          - tools/deploy/manifests/src/deployments

        newRootPath: "tools/deploy/manifests"

deployments should not be in there.

• using tools/deploy/manifests/src/deployments/**/* suddenly works.

[github-actions[bot]]

This issue is being marked as stale due to a long period of inactivity and will be closed in 5 days if there is no response.

[gabyx]

Shouldnt be closed as its a bug IMO (?)

[gabyx]

@joaopapereira : The following reproduces the error:

apiVersion: vendir.k14s.io/v1alpha1 kind: Config directories:

• path: t contents:

  • path: . git: url: https://gitlab.com/data-custodian/custodian.git ref: 85116b6dcf1e9224f2728c879451502fb5317b9f #depth: 1

    includePaths:

    • tools/deploy/manifests/src/**/* excludePaths:
    • tools/deploy/manifests/src/deployments

    newRootPath: "tools/deploy/manifests" deployments should not be in there.

• using tools/deploy/manifests/src/deployments/**/* suddenly works.

@joaopapereira : Can you check this bug again, and if it persiste, reopen the ticket, thanks for you work!