roaring icon indicating copy to clipboard operation
roaring copied to clipboard

Published 20 hours ago • verified publisher icon RoaringBitmap

Running unit tests downstream fails

Open anacrolix opened this issue 3 years ago • 1 comments

Running tests using a downstream project with go test all gives:

==roaring==
{1,2,3,4,5,100,1000}
{3,4,1000}
{}
Cardinality:  7
Contains 3?  true
1
3
4
5
1000

Wrote  22  bytes
I wrote the content to a byte stream and read it back.
size before run optimize: 1810 bytes, and after: 38 bytes.
2021/09/11 16:24:26 rtest N= 15
2021/09/11 16:24:26 rtest N= 1024
2021/09/11 16:24:26 rtest N= 4096
2021/09/11 16:24:26 rtest N= 65536
2021/09/11 16:24:26 rtest N= 1048576
2021/09/11 16:24:27 rtest N= 15
2021/09/11 16:24:27 rtest N= 100
2021/09/11 16:24:27 rtest N= 512
2021/09/11 16:24:27 rtest N= 1023
2021/09/11 16:24:27 rtest N= 1025
2021/09/11 16:24:27 rtest N= 4095
2021/09/11 16:24:27 rtest N= 4096
2021/09/11 16:24:27 rtest N= 4097
2021/09/11 16:24:27 rtest N= 65536
2021/09/11 16:24:27 rtest N= 1048576
2021/09/11 16:24:38 rtest N= 15
2021/09/11 16:24:38 rtest N= 1024
2021/09/11 16:24:38 rtest N= 4096
2021/09/11 16:24:38 rtest N= 65536
2021/09/11 16:24:38 rtest N= 1048576
2021/09/11 16:24:40 rtest N= 15
2021/09/11 16:24:40 rtest N= 100
2021/09/11 16:24:40 rtest N= 512
2021/09/11 16:24:40 rtest N= 1023
2021/09/11 16:24:40 rtest N= 1025
2021/09/11 16:24:40 rtest N= 4095
2021/09/11 16:24:40 rtest N= 4096
2021/09/11 16:24:40 rtest N= 4097
2021/09/11 16:24:40 rtest N= 65536
2021/09/11 16:24:40 rtest N= 1048576
--- FAIL: TestSerializationToFile038 (0.00s)
    serialization_test.go:70: 
        	Error Trace:	serialization_test.go:70
        	Error:      	Received unexpected error:
        	            	open myfile.bin: permission denied
        	Test:       	TestSerializationToFile038
    serialization_test.go:75: 
        	Error Trace:	serialization_test.go:75
        	Error:      	Received unexpected error:
        	            	invalid argument
        	Test:       	TestSerializationToFile038
    serialization_test.go:76: 
        	Error Trace:	serialization_test.go:76
        	Error:      	Not equal: 
        	            	expected: int64(0)
        	            	actual  : uint64(0x1e)
        	Test:       	TestSerializationToFile038
    serialization_test.go:83: 
        	Error Trace:	serialization_test.go:83
        	Error:      	Received unexpected error:
        	            	open myfile.bin: no such file or directory
        	Test:       	TestSerializationToFile038
    serialization_test.go:91: 
        	Error Trace:	serialization_test.go:91
        	Error:      	Should be true
        	Test:       	TestSerializationToFile038
    serialization_test.go:87: 
        	Error Trace:	serialization_test.go:87
        	            				serialization_test.go:92
        	Error:      	Received unexpected error:
        	            	remove myfile.bin: no such file or directory
        	Test:       	TestSerializationToFile038
--- FAIL: TestSerializationBasic4WriteAndReadFile040 (0.00s)
    serialization_test.go:123: 
        	Error Trace:	serialization_test.go:123
        	Error:      	Received unexpected error:
        	            	open testdata/all3.classic: permission denied
        	Test:       	TestSerializationBasic4WriteAndReadFile040
    serialization_test.go:129: 
        	Error Trace:	serialization_test.go:129
        	Error:      	Received unexpected error:
        	            	invalid argument
        	Test:       	TestSerializationBasic4WriteAndReadFile040
    serialization_test.go:130: 
        	Error Trace:	serialization_test.go:130
        	Error:      	Not equal: 
        	            	expected: int64(0)
        	            	actual  : uint64(0xbbb8)
        	Test:       	TestSerializationBasic4WriteAndReadFile040
    serialization_test.go:135: 
        	Error Trace:	serialization_test.go:135
        	Error:      	Received unexpected error:
        	            	open testdata/all3.classic: no such file or directory
        	Test:       	TestSerializationBasic4WriteAndReadFile040
    serialization_test.go:142: 
        	Error Trace:	serialization_test.go:142
        	Error:      	Received unexpected error:
        	            	error in roaringArray.readFrom: could not read initial cookie: invalid argument
        	Test:       	TestSerializationBasic4WriteAndReadFile040
    serialization_test.go:143: 
        	Error Trace:	serialization_test.go:143
        	Error:      	Should be true
        	Test:       	TestSerializationBasic4WriteAndReadFile040
200100
--- FAIL: TestBitmap_FromBuffer (0.02s)
    --- FAIL: TestBitmap_FromBuffer/all3.classic_bitmap (0.00s)
        serialization_test.go:461: 
            	Error Trace:	serialization_test.go:461
            	Error:      	Received unexpected error:
            	            	open testdata/all3.classic: no such file or directory
            	Test:       	TestBitmap_FromBuffer/all3.classic_bitmap
        serialization_test.go:466: 
            	Error Trace:	serialization_test.go:466
            	Error:      	Received unexpected error:
            	            	error in roaringArray.readFrom: could not read initial cookie: unexpected EOF
            	Test:       	TestBitmap_FromBuffer/all3.classic_bitmap
FAIL
FAIL	github.com/RoaringBitmap/roaring	29.203s

I expect there's some way to tell unit tests to more reliably find their test data. There may also be a special directory name that will be included in packages/modules.

anacrolix avatar Sep 11 '21 06:09 anacrolix

Pull request invited!

lemire avatar Sep 12 '21 00:09 lemire

How this error can be resolved?

Akhilesh53 avatar Jan 16 '23 13:01 Akhilesh53

Upgrade to https://github.com/RoaringBitmap/roaring/releases/tag/v1.2.2

Closing this issue.

lemire avatar Jan 16 '23 15:01 lemire

There's a lot less failures, but I still get:

$ go test github.com/RoaringBitmap/...
?   	github.com/RoaringBitmap/roaring/internal	[no test files]
==roaring==
{1,2,3,4,5,100,1000}
{3,4,1000}
{}
Cardinality:  7
Contains 3?  true
1
3
4
5
1000

Wrote  22  bytes
I wrote the content to a byte stream and read it back.
size before run optimize: 1810 bytes, and after: 38 bytes.
IMPORTANT: For testing file IO, the roaring library requires disk access.
--- FAIL: TestSerializationBasic4WriteAndReadFile040 (0.00s)
    serialization_test.go:128:
        	Error Trace:	/Users/anacrolix/go/pkg/mod/github.com/!roaring!bitmap/[email protected]/serialization_test.go:128
        	Error:      	Received unexpected error:
        	            	open testdata/all3.classic: permission denied
        	Test:       	TestSerializationBasic4WriteAndReadFile040
--- FAIL: TestBitmap_FromBuffer (0.00s)
    --- FAIL: TestBitmap_FromBuffer/all3.classic_bitmap (0.00s)
        serialization_test.go:466:
            	Error Trace:	/Users/anacrolix/go/pkg/mod/github.com/!roaring!bitmap/[email protected]/serialization_test.go:466
            	Error:      	Received unexpected error:
            	            	open testdata/all3.classic: no such file or directory
            	Test:       	TestBitmap_FromBuffer/all3.classic_bitmap
FAIL
FAIL	github.com/RoaringBitmap/roaring	23.483s
ok  	github.com/RoaringBitmap/roaring/BitSliceIndexing	5.518s
--- FAIL: Test_tryReadFromRoaring32_File (0.00s)
    roaring64_test.go:2016: open r32: permission denied
--- FAIL: Test_tryReadFromRoaring32WithRoaring64_File (0.00s)
    roaring64_test.go:2061: open r32: permission denied
2023/01/23 14:49:38 rtest N= 15
2023/01/23 14:49:38 rtest N= 1024
2023/01/23 14:49:38 rtest N= 4096
2023/01/23 14:49:38 rtest N= 65536
2023/01/23 14:49:38 rtest N= 1048576
2023/01/23 14:49:39 rtest N= 15
2023/01/23 14:49:39 rtest N= 100
2023/01/23 14:49:39 rtest N= 512
2023/01/23 14:49:39 rtest N= 1023
2023/01/23 14:49:39 rtest N= 1025
2023/01/23 14:49:39 rtest N= 4095
2023/01/23 14:49:39 rtest N= 4096
2023/01/23 14:49:39 rtest N= 4097
2023/01/23 14:49:39 rtest N= 65536
2023/01/23 14:49:39 rtest N= 1048576
IMPORTANT: For testing file IO, the roaring library requires disk access.
FAIL
FAIL	github.com/RoaringBitmap/roaring/roaring64	29.867s
FAIL

anacrolix avatar Jan 23 '23 03:01 anacrolix

I'll produce workarounds but I am very uneasy about the whole thing.

Basically, you are asking us to disable tests that have to do with file IO because you want to run tests in context where you don't have file access.

I'll do it, but I think it is a bad idea.

lemire avatar Jan 23 '23 15:01 lemire

See https://github.com/RoaringBitmap/roaring/releases/tag/v1.2.3

I did it but I don't like it. It effectively disables tests and let the code 'pass' even though some tests could not be executed.

lemire avatar Jan 23 '23 16:01 lemire

Thanks it works now.

I'm also not sure about the best practices for running tests from downstream. It's clearly supported from go test, but breaks down with Go modules, as it's probably not reasonable to include certain data (like large test files) in the modules as they are distributed. I've since run into cases where failing downstream is quite reasonable. It would be fantastic if we had a reference or link to something discussing this problem but I can't find one.

anacrolix avatar Jan 23 '23 21:01 anacrolix

In some tests, we want to write a (small) file and read it back and make sure that everything works. You want your dependencies to run these tests because it can expose bugs that you would not otherwise catch.

There is maybe a cool way to support that 'downstream'. I also did some research but only found ugly hacks.

lemire avatar Jan 24 '23 00:01 lemire