Add backend to write *.f files

[imphil]

trafficstars

Many EDA tools support digesting file lists in *.f files. Having a (pseudo-)backend which generates these files would serve as stopgap for EDA tools which we don't support natively so far, and opens the door to more exploration.

Feature list (to be discussed):

• Optionally add defines, parameters and libraries to the file list (some tools support this, other don't)
• Optionally output absolute paths

Design considerations:

• Can we enable fusesoc to use this backend without needing to add this new backend to each core file?

[olofk]

The idea of a dummy backend comes up from time to time. The big issue with such a backend is that it clashes with conditionals. With that said, I still understand there are many reasons for getting a list of files for use outside of what FuseSoC/Edalize can handle. But there is a way to do that already. FuseSoC will always write the EDAM file to the work root. It should be easy enough to parse that to extract the information of interest. It's a bit hacky since it requires at least running the setup stage, but perhaps we should allow a CLI flag that stops before even Edalize is called. What do you think?

[sjalloq]

@imphil @olofk, I was just about to ask some questions related to this as I'm looking at doing this for our project. Can we resurrect this thread.

So far I've got a 'filelist' backend that generates a VC file (.f/.vc) from the EDAM file list. It seems easy to generate a list of absolute file paths and include directories and write them out to the work directory. But what I wanted to do was allow the user to configure where the file should be written, for example, output the .f file parallel to the parent .core file. However, it doesn't look like the information for this is passed to Edalize. Is that correct? Or can I get the path to the parent .core file somehow?

@olofk , what was your point about conditionals? I don't understand the issue here as I've not got a huge amount of experience looking at the internals.

Shareef.

[sjalloq]

Here's my first pass which will hopefully stimulate some discussion: https://github.com/sjalloq/edalize/commit/08c07fd42df7a787c13196a900a6b52d7d9d1a85

It requires an update to FuseSoC too: https://github.com/sjalloq/fusesoc/commit/4db03432acddfcc4a0c4c605782f62afdf852f38

[sjalloq]

@olofk , is it possible to force --no-export from within a core file?

[yrrapt]

I too would be interested in this feature. @sjalloq created a modified version we used on a previous project together which can be found here. But having the feature as part of the upstream tool would be preferred.

[olofk]

Sorry about the lack of feedback here. I would like to think a bit differently about the problem actually. The main use case here seems to be to run tool flows that accept .f files but don't have an Edalize backend.

What I would like to do is to split this into two problems.

1. An EDAM to .f converter
2. A simple way to create custom flows

Looking at the first problem, I don't think this would necessarily have to be an Edalize backend. It can be a small utility that can be called on the command line or instantiated as a python class. This would allow stand-alone usage.

For the second problem, I propose we create a "custom" backend. The only options we give to that backend would be which commands to run during configure, build and run. We can then tell it to run the edam2f script during the configure stage and launch whatever EDA tool we want during the build phase.

What do you think?

Also, regarding conditionals, this is less of a problem now with the new Edalize architecture where a "tool" will not necessarily correspond to a "backend" as is the case now.

[olofk]

I had a vague memory of already doing something like this, and look what I just found https://github.com/olofk/edalize/tree/generic_tool

I suggest we clean that up by adding a configure_cmd option, dropping the flist stuff and create a standalone edam2f script instead that we ship as an executable together with edalize

[sjalloq]

Happy to look at this again in the future when I have time but for now we're already using this backend.

[olofk]

Great that you got something working at least. I'll ping you if I can find the time to massage your filelist backend into an edam2f tool

[sjalloq]

I'm happy to look at it if you want to send me some pointers and don't mind when it's done.

[shareefj]

@olofk Just wanted to resurrect this discussion.

You recently merged the Filelist backend that I opened a PR on (https://github.com/olofk/edalize/pull/289) but we both forgot that it required an accompanying change to Fusesoc which you objected to: https://github.com/olofk/fusesoc/pull/532

I never understood the objection and the thread went cold. Perhaps we can look at it again here and work out the best way forward.

[AWHoppe]

I and my team are very much interested in a filelist generation method for fusesoc/edalize!

This would help a lot in starting integrating the tools into current standards flows and keep things running before adopting them more.

I saw the last feature was removed and there's currently no branch for this feature, I would be interested in helping however/whenever possible.

[olofk]

Ok, so I added a preliminary flist tool now. Would be great if you could try it out before I push it to main. It's on the https://github.com/olofk/edalize/tree/flist branch

It uses the new Flow API and you can use it with the generic flow by setting the following keys in your target

targets:
  your_target:
    flow: generic
    flow_options:
      tool: flist

instead of the default_tool and tools keys

[shareefj]

Where does it write the filelist?

[olofk]

It gets written to the work root. If you are using FuseSoC, this is build/<flattened VLNV>/target by default, unless --work-root is used to override this.

[shareefj]

Can we get some feedback from others about whether this fits their needs before you push this. It doesn't fit ours so we won't be using it. I'll tidy up the Python version issues in our branch and people can use that if they want. I'll just have to change the class name if you push this.

[AWHoppe]

It looks really good.

We don't have currently any requirements about where the filelist should be placed. It would be convenient to have the option to choose to place it in the IP folder, but currently this is good.

The only change I propose is making the paths in the filelist global, instead of local to the work-root. This makes it easier for anyone to move or copy the files as they wish without the need to additionally text-parse the outputs for something the tool should be able to give.

Relative paths would constraint some tools or scripts. The environment would have to run from inside build//filelist, which would defeat the flexibility the filelist option provides.

[shareefj]

@olofk I'm going to write a script that wraps this behaviour of the new flist flow and enables the filelist to be dumped to an arbitrary location.

Would you be happy having this installed as an entry point parallel to fusesoc? If so, is filelist or filelister or flister a good name. Alternatively, I can just create a separate package that people could pip install.

I need something that outputs the filelist at the same location as the core file.

[shareefj]

This diff shows a working example. I fixed flist to create absolute paths, which is required for a generic filelist that can be reused outside of Fusesoc, and then added a new method that could be installed as an entry point.

At the moment the fusesoc core show command doesn't output the core file name which would be required to name the flist file using the same name. Can we update Fusesoc to do that? Something like the following with either the full path or just the basename.

$ git diff
diff --git a/fusesoc/capi2/core.py b/fusesoc/capi2/core.py
index 5cf42ff..768a843 100644
--- a/fusesoc/capi2/core.py
+++ b/fusesoc/capi2/core.py
@@ -516,6 +516,7 @@ class Core:
 Name:        {}
 Description: {}
 Core root:   {}
+Core file:   {}

 Targets:
 {}"""
@@ -539,6 +540,7 @@ Targets:
             str(self.name),
             str(self.get_description() or "<No description>"),
             str(self.core_root),
+            str(self.core_file),
             targets,
         )

flist.diff.txt