Specifying remote cores

[bluewww]

trafficstars

Let's say I have a testbench (a core itself) that requires three cores, each one not listed in the standard library but on available github. Each such core on github has an appropriate local dot core file.

What I want to do now add those cores as dependencies to the testbench core file, but as I understand it, fusesoc will (when appropriately told to) search recursively for dot core files in the current folder and in the standard library to resolve dependencies. Now since I don't have anything in the standard library it will search for all dot core files recursively.

What I now really want to tell fusesoc is that it can follow the dependency listed in the testbench core file (maybe a github link) and that it will there find the IP plus the dot core file. What I have to do instead is fully specify all dependencies in dot core files and add a provider section. Can I avoid this situation? Do I completely misunderstand the workflow that is supposed to be employed by using fusesoc?

[wallento]

Hi, I was about to propose something in a similar direction and thus want to add something more to your suggestion. @olofk @imphil I came to think about it around the discussion we had repeatedly about the LibreCores API. I feel it would be much simpler to have the FuseSoC side use remote git repositories. LibreCores would curate its own fusesoc_cores repository then instead of providing the core file locations via an API.

In the spirit of this and @bluewww's proposal, I would like to implement (or support someone who does it) the following two features:

1. Allow specification of remote repositories wherever paths are allowed (config file, command line, $FUSESOC_CORES, ...). npm has git://github.com/<user>/<project>.git#<branch>, pip has git://git.example.com/MyProject#egg=MyProject, so something like that would be great.

2. Allow a minimal core-file directive that allows to link vlnv names to a git reference. I think it should even be considered to have another file type for those ("core listing").

That is just a rough idea so far, parts of it I assumed should be there or have been there. I think those two cover the proposal above, plus they would enable us to keep centralized, curated lists of cores. Also, the default "fusesoc_core" can then be extended with e.g. git+https://github.com/librecores/fusesoc_cores

[wallento]

What fusesoc init does relates to this. I am not sure if it makes sense to keep it as it is. The mechanism of caching a remote repository (and wiping the cache) should be more generic IMHO.

[wallento]

Finally, sorry @bluewww is that along the lines what you meant? :)

[bluewww]

@wallento Thanks for the detailed information. I'm quite new to fusesoc so I'm not sure I entirely understand your proposal, so let my re-express my thoughts again:

• For each git repository containing an IP I'd like to maintain a dot core file.
• dot core files should be able to refer (depend) to other IPs sitting on git repos on github/gitlab/etc. Fusesoc then should try to resolve the IP by following the link and look for a dot core file there. This avoids fully specifying (listing all files) remote cores. I suppose this is covered by allowing vlnv names to be linked to git references.

Now as I understand, this seems to be a different model than what fusesoc seems to support. To get similar behaviour in the current version I'd either have to

• go to each git repository that is a dependency of my testbench and add it as library to fusesoc

or

• upstream a remote dot core file on https://github.com/fusesoc/fusesoc-cores. I don't think I can convince people to do a PR whenever they substantially change their IP. Editing a file local to the repository on the other hand is no issue.

[wallento]

Yes, you are right about your summary. I fully agree to your opinion. My work is going into a similar direction:

1. people maintain their local core files
2. they add their core to librecores.org
3. the librecores site maintains a fusesoc_cores repository
4. Using the librecores remote and a simple vlnv dependency will work then

For 4 we need some way of mapping and it has a nearly complete overlap with your idea. So, what I would need (because of the remote deflection repository) is something like:

map:
  - name: vendor1:library1:name1
    repo: git+https://github.com/vendor1/mylibrary@master
  - name: vendor2:library2:name1:1.0
    repo: git+https://github.com/vendor2/[email protected]

Such a map may then be used locally too so that you can simply use vlnv as before. Do you think it would be much better to also add the ability to add such a mapping in the dependencies directly?

[olofk]

Thanks for kicking off this discussion. There are several things being talked about here, but for the most basic task I think we are all on the same page. I have been considering to add an optional file to each library that contains just the provider section, very much like the map that @wallento describes (I do have some extra ideas for features though, but that can wait).

Now, there's a slighly different workflow that you perhaps can use as a workaround until this is in place. In your toplevel core, you can drop in the .core files or the dependencies and add a provider section to them. That way you only need to handle your toplevel core yourself, and FuseSoC will download and cache the dependencies

[olofk]

Oh, and regarding fusesoc init, until there's a remote API that FuseSoC can talk to, I think it makes sense to have an easy way to initialize the standard libraries

[olofk]

Also just realized I had already discretely opened an issue for this https://github.com/olofk/fusesoc/issues/269 :)

[wallento]

Yeah, it would probably mean adding or replacing a git remote path into the REPO variable in init