eval-dev-quality icon indicating copy to clipboard operation
eval-dev-quality copied to clipboard

`InstallToolsPath` is not used for test execution (`make test`)

Open Munsio opened this issue 1 year ago • 5 comments

The binary installed in ~/.eval-dev-quality/bin is not set in the PATH variable for the current execution resulting in all calls to the symflower binary failing with binary not found in PATH.

Either overwrite tools.SymflowerPath when command.InstallToolsPath is used or add command.InstallToolsPath to the PATH variable in the current execution context.

Munsio avatar May 06 '24 09:05 Munsio

Which binary? The eval binary? I currently use an .envrc file

#!/bin/bash

source ~/symflower/.envrc

export GO111MODULE=on
export PATH=$PWD/bin:$PATH

AFAIK @bauersimon as well. For now that is good enough to move forward. We have it on our list to install Go, maybe then we can set up a whole new DevEnv for everyone using symflower-make? Thoughts? But anyway. Fullly automatic DevEnv for this repo is not that important. When a user follows https://github.com/symflower/eval-dev-quality?tab=readme-ov-file#installation the will install Go usually in a way where the $GOPATH/bin directory is in PATH already.

@Munsio makes sense? Or do you mean something else? If that answers all questions, let's close.

zimmski avatar May 06 '24 10:05 zimmski

We are installing the symflower binary in https://github.com/symflower/eval-dev-quality/blob/main/cmd/eval-dev-quality/cmd/evaluate.go#L185 when we are running an evaluation and for that we are using a helper for the path https://github.com/symflower/eval-dev-quality/blob/main/tools/tools.go#L13.

But inside the https://github.com/symflower/eval-dev-quality/blob/main/model/symflower/symflower.go#L34 symflower model we totally ignore that path.

So everyone who has symflower not installed, checking out the the repo is not able to run it until either adding ~/.eval-dev-quality/bin to their PATH or installing symflower via the install scripts.

Munsio avatar May 06 '24 10:05 Munsio

What command/test fails for you? The eval-dev-quality command works for me when i remove the "symflower" binary.

Screenshot_20240506_124805

Can you show me how it fails for you?

zimmski avatar May 06 '24 10:05 zimmski

Ahhh okay, so only the tests are failing because of the missing symflower binary.

Alright, running the evaluation directly does work as expected but running the tests with make test will fail with symflower not found in PATH errors.

Munsio avatar May 06 '24 10:05 Munsio

Ah yes! So removing symflower binaries and then running make test does lead to a failure. We do not want to have the "installing" path everywhere. Concurrent access will be a nightmare.

zimmski avatar May 07 '24 06:05 zimmski