remove manual source in favor of using directory source

[Zebradil]

If vendir sync fails, manually managed directories might be deleted.

There are two paths in the config: 1) manually managed lib/manifests, 2) Non-existing git lib/bin

apiVersion: vendir.k14s.io/v1alpha1
kind: Config
directories:
- path: lib
  contents:
  - path: manifests
    manual: {}
  - path: bin
    git:
      url: https://github.com/k14s/vendir
      ref: origin/master
    includePaths:
    - bin/non-existing
    newRootPath: bin

Dir structure before sync:

.
├── lib
│  └── manifests
│     └── tpl.yaml
└── vendir.yml

After sync:

.
├── lib
└── vendir.yml

As far as I understand, vendir processes paths in the order they're defined in the configuration. For manually managed paths it moves them into temporary space. If one of the next paths can't be ensured due some error, the path previously moved to the temporary space is not moved back.

If I change the order of the paths, lib/manifests directory stay intact.

It looks like a bug and might lead to data loss (files and staged changes can be restored from git or from IDE history, but still).

[cppforlife]

@Zebradil yeah, manual source was very early addition that doesn't quite fit with methodology of vendir (vendir doesnt want to be "keeping" authoritative content). i would recommend switching to directory source. ill probably remove manual in the next release given that it can present very rough experience.

[Zebradil]

Alright, thanks for the update!

It seems like vendir works better within a dedicated directory. And I think I was trying to do something outside of the idea behind it.

My idea was to inject some third-party resources into my project without using a dedicated directory for that:

./infrastructure/kubernetes
├── bin                 # vendir: some executables from a common repo
│  └── ...
├── Makefile            # vendir: Makefile is the same across projects
├── manifests           # manual: rendered templates which will be applied by CI
│  └── ...
├── README.md           # manual
└── ytt
   ├── lib              # vendir: shared YTT templates from a common repo
   │  ├── admin
   │  │  └── ...
   │  ├── overlay
   │  │  └── ...
   │  └── apps          # vendir: selective sync apps from a common repo
   │     ├── 10-postgres
   │     │  └── ...
   │     └── 10-rabbitmq
   │        └── ...
   ├── local            # manual: specific stuff for the current project
   │  └── myapp
   │     └── ...
   └── values           # manual: project specific values [and overlays]
      ├── default.yaml
      └── secrets.yaml

Now I'm going to put vendir-managed stuff to a dedicated directory and maybe symlink Makefile from there. But anyways, separation of manual and vendir-managed resources looks more intuitive.

[cppforlife]

based on your breakdown above i dont think you would be able to do "manual" on the individual files so that already wouldnt work out.

but you should be able to have vendir.yml (at the root) with multiple top level directories for bin/, ytt/lib/

apiVersion: vendir.k14s.io/v1alpha1
kind: Config
directories:
- path: bin/
  contents:
  - path: .
    git:
      url: https://github.com/corp/shared-k8s
      ref: ...
    includePaths:
    - bin/**/*
    newRootPath: bin

- path: ytt/lib/
  contents:
  - path: .
    git:
      url: https://github.com/corp/shared-k8s
      ref: ...
    includePaths:
    - ytt/lib/**/*
    newRootPath: ytt/lib

though may be we should even support additional:

- path: Makefile # TODO support individual file?
- path: README.md # TODO support file existance?
- path: manifests/ # TODO support directory existance?

(could even be convenient to sync LICENSE, NOTICE, other misc files across various repos)

[cppforlife]

i do like a possibility of what you trying to do with manual though. probably should do more thinking around this...