GCC flags tuner should not use hardcoded temporary paths

[jbosboom]

The flags tuner has a --output option to select where the tuner writes temporary files, but it actually uses various directories under ./tmp regardless of the value of this option. This causes multiple instances of the tuner to interfere with one another.

This should be fixed, and the default should use the pid or datetime to generate a unique (if uninformative) pathname by default, so things just work.

[jbosboom]

I propose to split this into two or three options:

• --temp-dir: usually /tmp or ./tmp; we expect users to set this for all projects on a particular system.
• --temp-subdir-format: default {node}/{datetime}-{pid} to ensure uniqueness across tuning runs (from any project). Users shouldn't need to change this, so we could get away with hardcoding it to prevent it being an attractive nuisance.
• --output: default gccflags.bin; the filename we compile to, also used in the final flags command file. We expect users to set this per project they're tuning. We'll append -{node}-{pid}-{datetime}.{cmd, json} to generate the final configuration files, or make it another option (see #67), to get uniqueness across multiple tuning runs of the same project.

There are essentially three scopes here: machine settings for all projects (--temp-dir), project settings (--output, which will often be the project name), and tuning runs (handled automatically, or via --temp-subdir-format).

[jansel]

For the first two, we should just use https://docs.python.org/2/library/tempfile.html

This will respect OS-specific environment variables to find temporary storage.

On Thu, Sep 10, 2015 at 7:08 PM, Jeffrey Bosboom [email protected] wrote:

I propose to split this into two or three options:

• --temp-dir: usually /tmp or ./tmp; we expect users to set this for all projects on a particular system.
• --temp-subdir-format: default {node}/{datetime}-{pid} to ensure uniqueness across tuning runs (from any project). Users shouldn't need to change this, so we could get away with hardcoding it to prevent it being an attractive nuisance.
• --output: default gccflags.bin; the filename we compile to, also used in the final flags command file. We expect users to set this per project they're tuning. We'll append -{node}-{pid}-{datetime}.{cmd, json} to generate the final configuration files, or make it another option (see #67 https://github.com/jansel/opentuner/issues/67), to get uniqueness across multiple tuning runs of the same project.

There are essentially three scopes here: machine settings for all projects (--temp-dir), project settings (--output, which will often be the project name), and tuning runs (handled automatically, or via --temp-subdir-format ).

— Reply to this email directly or view it on GitHub https://github.com/jansel/opentuner/issues/71#issuecomment-139429941.