Share project git repository across plans

[psss]

Currently the discover step always creates in the workdir a new copy of the project git repository for each plan. It would make sense to copy the repository only once and use symlinks from individual plan workdirs to safe disk space.

[thrix]

yeah, makes sense to me, currently I do not see any problems doing that

[thrix]

Would be nice to unify somehow the directory structure for saving such data, feels a bit like rpms, maybe other artifacts needed by other steps could share this unified space together

[pvalena]

I'd prefer to avoid this, as the runtime can modify those cloned-out repositories, right? If one runs make, in some prepare phase, and it runs in that directory, it's contents is modifired, right?

To me it sounds like worktree has the functionality we need (instead of symlinks). You can think of it like git symlink.

To elaborate: In case we'd want to manage the repo clone "ourselves", we'd have to always store,and reset the commit for that repo (runs can have different commits). And always remove all untracked files before some run (and store+track those modifications / untracked files for the previous runs). But that's IMO like reinventing the worktree I've mentioned above.

AFAIR the only limitation is that you shouln't push, or commit into same branch in different worktrees (not sure how those conflicts are created / resolved, I just recieved that guideline from git maintainer:)).

[pvalena]

Would be nice to unify somehow the directory structure for saving such data, feels a bit like rpms, maybe other artifacts needed by other steps could share this unified space together

It's hard for me to imagine other usecases - what steps would benefit from this caching?

[thrix]

@pvalena did not think about that (make example) I guess that makes sense, feels dangerous! Thanks @pvalena

So I take my good idea of this back ...

@psss seems this would break cases where people for example build in each plan differently the git repo, etc.

[pvalena]

Let me reiterate, in case I didn't write it clearly enough in my previous comment:

• In general this is a good idea
• please do not use symlinks
• please do not implement our-own-caching-tm
• let's use worktree functionality which does exactly what we need

[psss]

Thanks for bringing the idea of using worktree for this. Seems this could be quite useful although we would not save as much disk space as we would if using symlinks. On the other hand, there really could be unexpected results if several plans are modifying the same work tree. Especially if we plan to support a parallel execution of plans.

Was playing a bit with the feature and I see one possible limitation: It seems it's not possible to checkout the same branch into multiple worktrees:

> git checkout master
fatal: 'master' is already checked out at '/tmp/tmt'

But, perhaps, for our use cases using a detached worktree would / could be enough?

[psss]

Closing in favor of #423 which has a nice outline of the solution and is already assigned by @thrix.

[psss]

Oh, I forgot we still want to cover the following here:

• discover step should use a symlink if url and ref are not provided

This is blocked by #423. Sorry for the spam...

[thrix]

@psss I guess that can be part of #423 also ... seems I will be a bit area also there