Broken "version" attribute in git provider

[JLeemaster]

Summary

Specifying a version attribute when using the git provider is currently broken as implemented. the root cause was introduced by commit 8911f7c.

The current implementation will perform git clone with a --depth 1 argument. A side effect of cloning with the --depth 1 flag is that all tags & branch references that are not the current HEAD will be omitted from the cloned repo. Therefore, if the git provider's version attribute is set to any tag or branch other than the current HEAD of the repository, it will fail when trying to checkout the branch or tag specified in the version attribute.

How to reproduce

1. Clear the fusesoc cache
2. Take any known-good .core file that uses the github provider and specifies a version that is not the most recent master or main commit for that repository
3. Convert the .core file to use the git provider instead of the github provider
4. Attempt to use it with fusesoc to download the core

Suggested solution

After the git provider executes the git clone --depth 1 command:

1. Retrieve the list of available references from the remote repo
   • e.g. git ls-remote origin
2. Scan the returned list for a branch or tag that matches version
3. If a match is found, or if version is a fully-specified SHA1 commit ID, fetch it accordingly
   • if it's a tag:
     • fetch it and make it a local tag
       • git fetch --depth 1 origin refs/tags/${version}:refs/tags/${version}
   • if it's a branch or a fully-specified SHA1 commit ID:
     • fetch it normally
       • git fetch --depth 1 origin ${version}
4. checkout the fetched branch, tag, or commit ID
   • If it is a branch or a tag:
     • git checkout ${version}
   • if it is a full SHA1 commit ID:
     • git checkout FETCH_HEAD // detached-head checkout of whatever was fetched

This will keep the benefits of doing a shallow git clone while still being able to specify branches, tags, & commit IDs that are not the current HEAD of the remote repo as the core's version.

[mithro]

Maybe the stuff at https://github.com/YosysHQ/yosys/pull/3138 could help?

[olofk]

Thank you for the report and suggested solution. Someone else noticed this issue in the chat recently. I thought the problem was that we did a clone and then a checkout afterwards and that we could just simply do both in one step. I hadn't realized that git by default only know about the tags available in HEAD. That does complicate things.

@mithro That yosys issue seems to be another problem.

[GCHQDeveloper211]

Hi @olofk,

We have just found this issue too, and came to a solution before seeing this ticket.

Our fix also maintains the speed of a shallow clone but fixes the issue. We do this by adding the --no-single-branch arg to the clone command in provider/git.py on line 40, having the resulting line being args = ["clone", "-q", "--depth", "1", "--no-single-branch", repo, local_dir]

[olofk]

Perfect. Seems like this is fixed with #585. Please reopen otherwise

[JLeemaster]

I think this closes the issue as well but will give it a test once I get a chance.