couchdb
couchdb copied to clipboard
Test failures on s390x architecture
Running CouchDb 3.x on s390x shows a bunch of test failures seemingly related the compiling javascript code with libmozjs-60-0:s390x
package.
$ docker run -it apache/couchdbci-debian:s390x-buster-erlang-20.3.8.26-1
$ cd /home/jenkins
$ git clone https://github.com/apache/couchdb.git
$ ./configure --spidermonkey-version 60 --dev
$ make
$ make eunit apps=couch_mrview suites=couch_mrview_all_docs_tests
Here is an example of a test failure and how to run it:
==> couch_mrview (eunit)
======================== EUnit ========================
module 'couch_mrview_all_docs_tests'
_all_docs view tests
couch_mrview_all_docs_tests:70: should_query...*failed*
in function couch_mrview_all_docs_tests:'-should_query/1-fun-0-'/2 (test/eunit/couch_mrview_all_docs_tests.erl, line 70)
**error:{assertEqual,[{module,couch_mrview_all_docs_tests},
{line,70},
{expression,"Result"},
{expected,{ok,[{meta,[{total,11},{offset,0}]},
{row,[{id,<<"1">>},{key,<<"1">>},{value,{...}}]},
{row,[{id,<<"10">>},{key,<<...>>},{value,...}]},
{row,[{id,<<...>>},{key,...},{...}]},
{row,[{id,...},{...}|...]},
{row,[{...}|...]},
{row,[...]},
{row,...},
{...}|...]}},
{value,{ok,[{meta,[{total,10},{offset,0}]},
{row,[{id,<<"1">>},{key,<<...>>},{value,...}]},
{row,[{id,<<...>>},{key,...},{...}]},
{row,[{id,...},{...}|...]},
{row,[{...}|...]},
{row,[...]},
{row,...},
{...}|...]}}]}
output:<<"">>
couch_mrview_all_docs_tests:80: should_query_with_range...*failed*
in function couch_mrview_all_docs_tests:'-should_query_with_range/1-fun-0-'/2 (test/eunit/couch_mrview_all_docs_tests.erl, line 80)
**error:{assertEqual,[{module,couch_mrview_all_docs_tests},
{line,80},
{expression,"Result"},
{expected,{ok,[{meta,[{total,11},{offset,3}]},
{row,[{id,<<"3">>},{key,<<"3">>},{value,{...}}]},
{row,[{id,<<"4">>},{key,<<...>>},{value,...}]},
{row,[{id,<<...>>},{key,...},{...}]}]}},
{value,{ok,[{meta,[{total,10},{offset,3}]},
{row,[{id,<<"3">>},{key,<<...>>},{value,...}]},
{row,[{id,<<...>>},{key,...},{...}]},
{row,[{id,...},{...}|...]}]}}]}
output:<<"">>
...
[done in 0.336 s]
[done in 1.609 s]
=======================================================
Failed: 5. Skipped: 0. Passed: 0.
The logs are full of these errors:
**throw:{compilation_error,<<"(new SyntaxError(\"missing } in compound statement\", \"<unknown>\", 3))"...>>}
output:<<"">>
Hi @nickva I tried executing same steps mentioned by you however make itself failing at my end.
$ make
==> couch_epi (compile)
ERROR: OTP release 20 does not match required regex 21|22|23|24
ERROR: compile failed while processing /home/jenkins/couchdb/src/couch_epi: rebar_abort
Could you please share
- output of configure and make command
- if any other dependency/configuration is used
@Nayana-ibm
Thanks for taking a look. Sorry I forgot to mention that this is for the 3.x branch only. So before the configure step, we'd switch to the 3.x branch.
git checkout 3.x && ./configure --dev --spidermonkey-version 60
...
FoundationDB which is needed for the main
branch (the future 4.x) can't compile on either ppc64le or s390x yet.
Few observations: Checked apache/couchdbci-debian:buster-erlang-20.3.8.26-1 for amd64 and could see package couch-libmozjs185-dev installed However on s390x image, couldn't find this package.
Tried building and installing erlang 22.2, elixir 1.10.2, node v14.2.0 and spider monkey js-1.8.5 on ub18.04. Test are passing with couchdb 3.x with this.
Good observation @Nayana-ibm. We use libmozjs185 on ppc64le so perhaps there some issue with the newer JS library and we should switch to libmozjs185 for s390x for the time being.
@nickva So what is the plan you suggest to proceed further on this ?
As @Nayana-ibm has pointed, couch-libmozjs185-dev
which is installed on apache/couchdbci-debian:buster-erlang-20.3.8.26-1
but not available on apache/couchdbci-debian:s390x-buster-erlang-20.3.8.26-1
.
Can we build the js185 on s390x using the steps specified here - https://github.com/linux-on-ibm-z/scripts/blob/master/CouchDB/3.1.1/build_couchdb.sh#L170 ?
OR
Should we have s390x version of couch-libmozjs185-dev
package built similar to how its done on amd64 ?
Ideally we'd find why mozjs60 (spidermonkey version 60) doesn't work on s390x but for the time being we should probably try to build as a deb/rpm package just like we build it for amd64 and ppc64le.
The best place to start is with the readme file in https://github.com/apache/couchdb-pkg
Then https://github.com/apache/couchdb-pkg/search?q=ppc64le shows how ppc64le is handled, I imagine s390x would look similar.
Sorry to bother @Nayana-ibm, not sure if you'd even know about it, but it looks like the s390x Jenkins worker donated by IBM (couchdb01
) is down.
To help identify it, the IP can be seen in: https://github.com/apache/couchdb-infra-cm/blob/02a0d1ed2dbbebf2b88d5fcf4e94d9022acd2362/production#L167-L179
Did it accidentally got shut down or removed and just needs a restart or is it gone for good?
s390x node was removed from the CI workers lists for now https://github.com/apache/couchdb-infra-cm/commit/67a646854cef7e634855390f9540e0922e83f1f3
With the later OS variants and a new KVM based s390x CI instances all tests now pass the s390x is part of the regular full CI run.