emp-tool icon indicating copy to clipboard operation
emp-tool copied to clipboard

Separate benchmarks from tests

Open weikengchen opened this issue 3 years ago • 1 comments

This is related to the high overhead of CI.

It appears that certain tests in the repo are closer to a benchmark than a test.

A good example is here:

	for (long long length = 2; length <= 8192; length*=2) {
		long long times = 1024*1024*32/length;
		block * data = new block[length+1];
		char * data2 = (char *)data;
		auto start = clock_start();
		for (int i = 0; i < times; ++i) {
			prg.random_data(data2, length*16);
			//prg.random_data_unaligned(data2+1, length*16);
		}
		double interval = time_from(start);
		delete[] data;
		cout << "PRG speed with block size "<<length<<" :\t"<<(length*times*128)/(interval+0.0)*1e6*1e-9<<" Gbps\n";
	}

This is left as a future todo. It is also challenging to think about what tests would give us good coverage.

weikengchen avatar Dec 27 '20 06:12 weikengchen

github action added, with a self-hosted machine for ARM. I think one option is to take the test size as input. That will solve the problem. (I don't want too many files:P

wangxiao1254 avatar Feb 10 '21 19:02 wangxiao1254