keylime icon indicating copy to clipboard operation
keylime copied to clipboard

CI testing improvements for run_local.sh

Open lkatalin opened this issue 1 year ago • 2 comments

It would be easier to read the output and also faster to rerun tests if we could specify a subset of the tests to run.

lkatalin avatar Sep 19 '22 15:09 lkatalin

It is also not very clear when tests on the Python agent stop and tests on the Rust agent begin.

lkatalin avatar Sep 19 '22 16:09 lkatalin

Does captured stdout refer to the output directly above or below?

lkatalin avatar Sep 19 '22 16:09 lkatalin

@lkatalin @mpeters @THS-on @ansasaki @stefanberger Just wanted to confirm that we plan to keep maintaining run_local.sh (which is broken for both rust and python agents at the moment). My vote is to keep it, but I am open to sunsetting it the majority decides it is not worth the trouble.

maugustosilva avatar Nov 07 '22 20:11 maugustosilva

Would it make sense to drop or rework the tests that seem broken or no longer applicable? It seems good to have a way to run integration tests locally, but those could (and maybe should) look very different from the current run_local.sh.

lkatalin avatar Nov 07 '22 20:11 lkatalin

There should be an easy way to run all these tests locally.

When it comes to the rust agent, it should be possible to cache the results, maybe by checking the rust agent out into (hidden) directory of the local keylime checkout because watching it download and build the Rust agent for several minutes every time one runs the local tests isn't very productive.

stefanberger avatar Nov 07 '22 21:11 stefanberger

I think @ansasaki had some thoughts on why it was broken and some potential fixes. I agree that we should keep it and always have a way of running the tests locally.

mpeters avatar Nov 07 '22 21:11 mpeters

I believe PR #1325 basically fixed it. I will close it for now, feel free to open in case you believe it still needs more work.

maugustosilva avatar Mar 31 '23 20:03 maugustosilva