Running two targets for the same core simultaneously without `--no-exports` results with error.

[m-kru]

This behavior results from 2 facts:

1. When sources are exported, they are copied to the src/ directory which is located in the build/core/ directory, not in the build/core/target/ directory.
2. On the new export for the same core, but different target, the src/ directory is removed (capi2/core.py file).

    def export(self, dst_dir, flags={}):
        if os.path.exists(dst_dir):
            shutil.rmtree(dst_dir)

Sources which are included in the first target, but are not included in the second one, will be removed and run for the first target will fail. This makes it impossible to run some verification targets in parallel.

One can say, "so lets run them with --no-export flag". However, some verification targets may take significant time, and in the meantime user may want to continue working.

I would like to make the src/ directory, with copied sources, to be bound to a target, not a core. Does anyone see any potential problems?

[ThomasHornschuh]

Why not get a step further and have an option to run FuseSoC on a temporary target directory with a unique name for every run? This would give the option to run several configurations (e.g. different parameters) of the same target in parallel.
A common case would be a CPU test suite, where you usually run a lot of small test programs in the CPU core.

I tried this a while ago with RISC-V compliance and was already thinking about such a feature for FuseSoC, but then postponed the idea because the makefiles in the compliance suite itself are not built in a way to support parallel runs with using the -j option of make.

[imphil]

@m-kru I need to think about that a bit more, but I see your point.

In the mean time, the easiest solution is to use --build-root /some/unique/directory (such as --build-root "$(mktemp -d)")

[m-kru]

@ThomasHornschuh I think I do not understand which configurations you have in mind. In case of .core file you define parameters per target, so each target already has distinct configuration. Example snippet:

targets:
  base_target: &base
    default_tool: vivado
    tools:
      vivado:
        part: xc7k325tffg900-2
    filesets:
     - vivado_project_settings
     - common

  shuffler_sequential_3_muxes:
    <<: *base
    toplevel: top_shuffler_sequential
    filesets_append:
      - top_shuffler_sequential
    parameters : [G_NUM_MUXES=3]

  shuffler_sequential_4_muxes:
    <<: *base
    toplevel: top_shuffler_sequential
    filesets_append:
      - top_shuffler_sequential
    parameters : [G_NUM_MUXES=4]

  shuffler_sequential_5_muxes:
    <<: *base
    toplevel: top_shuffler_sequential
    filesets_append:
      - top_shuffler_sequential
    parameters : [G_NUM_MUXES=5]

parameters:
  G_NUM_MUXES:
    datatype : int
    description : Number of internal muxes in sequential shuffler.
    paramtype : generic

[ThomasHornschuh]

This is one way you can work, but will come to a limit if you have a lot of configurations or configurations which are defined externally. Are you aware that you also can pass the parameters to FuseSoC during invocation, so in your case with adding the --G_NUM_MUXES=n to the FuseSoC command line?

For example: https://github.com/bonfireprocessor/bonfire-cpu/blob/next/bonfire-cpu.core has a parameter TestFile which refers to the machine code to be executed during the test and a parameter signature_file which is the filename/path to write a memory dump of a specific memory section which is than compared with a golden reference.

So for every test FuseSoC is called with something like: fusesoc run --target=sim bonfire_cpu --TestFile=<....> --signature_file=<.....>

There are also other configuration parameters of the cpu core (e.g. branch_predictor on/off) which can be set this way. It is obvious that all the tests run independently of each other, and because in my case GHDL only runs single threaded sequential runs cannot saturate the machine.

Because FuseSoC also places the temporary runtime artifacts (e.g. object files) in the build directory, you cannot run several instances of FuseSoC in parallel. An automatically created temporary build directory would help in this case.

[m-kru]

Ok, I understand. In such case maybe it would be good to use timestamp as the extra, unique part of the target directory?

[Jbalkind]

We use --build-root all the time and it works well. Also keeps the naming choices out of fusesoc.

[ThomasHornschuh]

@m-kru

Ok, I understand. In such case maybe it would be good to use timestamp as the extra, unique part of the target directory?

Never use timestamps for unique names, it will fail in case where two creations happen at the same time , "same" in the sense of the resolution of the timestamp..

@Jbalkind

We use --build-root all the time and it works well. Also keeps the naming choices out of fusesoc.

I tend to agree, I only mentioned my use case, because I think that @m-kru 's proposal "jumps to short" and only handles a special case.

[m-kru]

Yeah, creating unique directory for each run should not be done by FuseSoc, as this can be easily achieved with --build-root. Plus this is something you do not need in most scenarios.

However, I still think that by default the src/ directory should be crated inside build/core/target, not build/core, as used file sets are chosen per target, not core.

[Jbalkind]

I seem to recall discussing this with Olof a couple of years ago and him predicting that this would come up at some point.

The environment requiring us to recall which target we built most recently before we look in src/ definitely seems like poor behaviour. I'm not convinced that there's enough of a duplication of code if we have per-target src/ for this to cause a disk space issue (particularly when you can just rm the other targets' src/ directories if needed). As such I'm for changing this, though I have minor concerns about maintaining backwards compatibility.

[ThomasHornschuh]

The current structure has the advantage that the runtime artifacts are clearly separate from the source code. So there can also be no naming conflicts between files and directories created by FuseSoc and the tool.

Regarding backwards compatibility: I have some cores which use e.g. tcl scripts to create block designs in Vivado. For this to work the relative relationships between the source files and the tool "runtime" directory must be stable.

To avoid breaking such types of cores a new tool specific directory structure should keep this relationship.

A solution could be to move everything which is today in the target specific directory to a a sub directory "run". Than source directories will than be located next to it also into the target directory. In this way the relative relationship stay the same

[olofk]

There is some history here. I originally thought it would be possible to have a common source directory with multiple target directories. Oh how young and foolish I was. It's possible to do it, but needs a lot of extra code which I don't think anyone will have time to write right now, so I have been planning for several years to move source under the target dir. This is also why I have been very clear that no one should make assumptions about the relative locations of the source and target dirs. There used to be scripts that referenced files from ../src/some_core/ but these should hopefully all be gone by now. Hopefully, the vast majority of these cases should now be covered by the file parameter type and the copyto flag for files

I think the reason why the change haven't been done yet is a mix of fear of breaking people's scripts and that --build-root helps in most cases. But if anyone wants to give a shot at moving the source under the target dir, I'm happy to accept such patches

I still think it would be useful to have an export mode where you can specify several tools, so that it's possible to generate archives of a whole design with tool scripts for multiple tools that can be used without FuseSoC. This would however need code to detect if there are any file conflicts or so. The easiest way is probably to do two exports and then some kind of merge.

[olofk]

Trying to figure out what remains in this issue report. One issue is definitely that the src dir is shared by all work roots by default and this just isn't a very good idea. I'll file a bug explcitly for that. But are there any other unresolved issues here or can we close the bug now?

[olofk]

Fixed with a4cb4a2a26e389ced0c904013c0574ec4e12e0b1