Specifying branch/tag and depth in git URI spec in fusesoc.conf

[aprnath]

Please add a mechanism to specify a branch/tag name and additional options like '--depth' to a library's git URI. For repositories that are large, with many branches and tags, without such a mechanism one has to interrupt the process after the cloning and manually switch to the appropriate branch/tag. Adding --depth allows for shallow clones.

Thanks!

[martin-lueker]

Hi @aprnath, I'm looking into this right now. A mechanism does exist for remote cores, and it seems that the github provider for remote cores should do the job, but right now it looks like it only works on github. I'm not sure it will work for vanilla git repos (like bitbucket).
So maintainers: I think this needs to be clarified that it's not enough to get this to work on just github, it has to be for any get repo (github, gitlab, bitbucket, etc).

[CohenSteed]

Additional encouragement for this feature - implementations might include multiple layers of hierarchy. To fully support precise reference, the designation of branch/tag as a requirement for a particular entry (assume master if unspecified?) would go a long way to put some guards on this process to keep the user from doing something unintended.

[olofk]

@martin-lueker FTR, the github provider is actually a special case of the URL provider. It uses Github's API for getting an archive of a specific version. I do this to minimize the amount of data to transfer. This is not a git feature per se, but it could be that e.g. gitlab offer a similar thing

@CohenSteed, we can already today specify an exact version which can be a tag, branch or a sha for a commit. Or did I misunderstand?

Ok, so I see a couple of levels of flexibility here. The least flexible would be to just add --depth 1 as a hard coded argument. I think that makes most sense as users should never modify the contents of remote cores anyway. In that case they should use local cores. So I would need a good argument for why offering the user to specify the depth by themselves. Always striving to minimize the number of things to configure.

Now, there's another related thing I'm wondering about. Is there a need to support submodules (is that what you were getting at @CohenSteed?). I dearly hope the answer is no and that everyone is sensible enough to understand that it's an awful way to handle dependencies, but again, perhaps I'm missing some good use case here

[olofk]

@martin-lueker FTR, the github provider is a special case of the URL provider rather than the git provider. It uses github's api to fetch an archive a specific version. I use this to minimize the data transfers

@CohenSteed You can already now specify a version, which can be a branch, tag or sha. Is that what you were looking for or did I misunderstand?

The simplest way to fix this is to hardcode --depth 1 in the call to git. Users shouldn't touch the data of the remote cores (if they do, they should use local cores instead) so I can't see a good reason for letting the users configure this by themselves. But maybe I'm missing a use case here.

In a somewhat similar vein, some users could be interested in doing recursive checkouts if they have submodules (is that what you meant with hierarchy, @CohenSteed ?). I really really hope this isn't the case though, and people just use FuseSoC's dependency resolution isntead of submodules. But again, perhaps I'm missing a use case

[CohenSteed]

@olofk Thank you, that's exactly what I was looking for - the direct specification of a branch. I see how it can be done, now, thank you.

For dependencies, I would expect a second (or nth) .core file to be used, which would also specify the relevant branch, if necessary. Perhaps layering is best achieved with .core -> .core linkage, with each .core specifying it's own local branch? That's how I'd do it, and it looks like that's how we're doing it now.

[olofk]

@CohenSteed Great. So I guess that your concerns are handled then, and I just need to know from @martin-lueker and @aprnath if a hardcoded depth=1 works fo their cases

[aprnath]

Olof, --depth=1 will certainly work. What about a specific branch name?

[martin-lueker]

Olof, --depth=1 will certainly work. What about a specific branch name?

If you are using remote cores, a branch name is already supported. Cohen and I can show you how.

[martin-lueker]

depth=1 is fine for me for now.

Maybe tangential:(?) Longer term I have some bigger frustrations, related to how the git repo and the build directory interact. I tend to spend a lot of time in active development mucking around in the build directory and then I have to copy things back to my repo at the very end. Maybe you have some better suggestions on how to manage that kind of issue.

[olofk]

Good. Then we have a fix @martin-lueker, for the other problem, have you tried --no-export ?

[martin-lueker]

--no-export... Ooh I've never noticed that. I will try that. Thanks! -Martin