Startup Problem

[Miyayx]

Hi,

I've got into trouble with Virtuoso7 when trying to start it. The virtuoso.log showed :

04:42:05 Compiler unit is timed at 0.000330 msec
04:42:05 A bad parent link has been detected by looking at the would be parent and finding no leaf pointer. The pages follow, parent first.
04:42:05 Bad parent link corrected
04:42:05 ./virtuoso() [0x97f6f1]
04:42:05 ./virtuoso() [0x42e4d1]
04:42:05 ./virtuoso() [0x4dbfc2]
04:42:05 ./virtuoso() [0x4dc53b]
04:42:05 ./virtuoso() [0x4dd1e0]
04:42:05 ./virtuoso() [0x4dd400]
04:42:05 ./virtuoso() [0x5b5799]
04:42:05 ./virtuoso() [0x5985a4]
04:42:05 ./virtuoso() [0x5963bb]
04:42:05 ./virtuoso() [0x5a2503]
04:42:05 ./virtuoso() [0x5a26a8]
04:42:05 ./virtuoso() [0x5a29aa]
04:42:05 ./virtuoso() [0x62232e]
04:42:05 ./virtuoso() [0x55e43f]
04:42:05 ./virtuoso() [0x55ebd2]
04:42:05 ./virtuoso() [0x581e2b]
04:42:05 ./virtuoso() [0x582ccf]
04:42:05 ./virtuoso() [0x581ca6]
04:42:05 ./virtuoso() [0x5825db]
04:42:05 ./virtuoso() [0x55e6c5]
04:42:05 ./virtuoso() [0x55f784]
04:42:05 ./virtuoso() [0x8eabb5]
04:42:05 ./virtuoso() [0x8f792f]
04:42:05 ./virtuoso() [0x61d4be]
04:42:05 ./virtuoso() [0x410f6d]
04:42:05 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7fac9198fec5]
04:42:05 ./virtuoso(__gxx_personality_v0+0x2b9) [0x40b0e9]
04:42:05 GPF: gate.c:1030 internal error

3 days before I deleted the virtuoso.trx and loaded several large ttl files into the database. This problem happened after I killed the Virtuoso process today. To fix it, I tried to change the AutoRepair==1 in Virtuoso.ini, but it still didn't work.

[HughWilliams]

What version of Virtuoso is being used here (virtuoso-t -?) , as if not already using the new 7.2 open source release we would recommend upgrading to it.

If "ulimit -c unlimited" is set before startup, does a core file get created from which a gdb back trace can be obtained to get more details on the cause of the error ?

"AutoRepair" is section in the INI file and not a param you set "BadParentLinks = 1" as indicated in the docs at:

http://docs.openlinksw.com/virtuoso/databaseadmsrv.html#ini_AutoRepair

Ultimately you may have to run a crash dump and recovery of the database as detailed at:

http://docs.openlinksw.com/virtuoso/databaseadmsrv.html#backup

Certainly and integrity check should be run on the database once it starts "backup '/dev/null';

[BobTheta]

Hi, Hugh, I enabled the core before startup Virtuoso7. And gdb shows:

Type "apropos word" to search for commands related to "word". [New LWP 5318] [New LWP 5319] [New LWP 5320] [New LWP 5321] Core was generated by `./virtuoso +wait'. Program terminated with signal SIGSEGV, Segmentation fault. #0 0x000000000097f78a in ?? ()

I can't understand it well.

best regards, Bob

[BobTheta]

And these: (gdb) bt #0 0x000000000097f78a in ?? () #1 0x00007fd8dd085d78 in ?? () #2 0x0000000000b6f57f in ?? () #3 0x000000000097f6f1 in ?? () #4 0x000000000042e4d1 in ?? () #5 0x00000000004dbfc2 in ?? () #6 0x00000000004dc53b in ?? () #7 0x00000000004dd1e0 in ?? () #8 0x00000000004dd400 in ?? () #9 0x00000000005b5799 in ?? () #10 0x00000000005985a4 in ?? () #11 0x00000000005963bb in ?? () #12 0x00000000005a2503 in ?? () #13 0x00000000005a26a8 in ?? () #14 0x00000000005a29aa in ?? () #15 0x000000000062232e in ?? () #16 0x000000000055e43f in ?? () #17 0x000000000055ebd2 in ?? () #18 0x0000000000581e2b in ?? () #19 0x0000000000582ccf in ?? () #20 0x0000000000581ca6 in ?? () #21 0x00000000005825db in ?? () #22 0x000000000055e6c5 in ?? () #23 0x000000000055f784 in ?? () #24 0x00000000008eabb5 in ?? () #25 0x00000000008f792f in ?? () #26 0x000000000061d4be in ?? () #27 0x0000000000410f6d in ?? () #28 0x00007fd8dd096ec5 in ?? () #29 0x000000000040b0e9 in ?? () #30 0x00007fd8de696128 in ?? () #31 0x00007fd8dd07ef80 in ?? () #32 0x00007fd8de6979e0 in ?? () #33 0x00007fd8de6aa1c8 in ?? () #34 0x0000000000000007 in ?? () #35 0x0000000000000006 in ?? () #36 0x00007fd8dd4341c0 in ?? () #37 0x0000000a6df73200 in ?? () #38 0x00000007d2e53f88 in ?? () #39 0x0000000a6e5a5258 in ?? () #40 0x00007fd8dd0c93f7 in ?? () #41 0x000000000325d6a0 in ?? () #42 0x0000004000000010 in ?? () #43 0x00007fff91e52e20 in ?? () #44 0x00007fff91e52d60 in ?? () #45 0x0000000000000000 in ?? ()

[BobTheta]

Because of the fail of startup. "backup '/dev/null' " didn't work.

SQL> backup '/dev/null'; *** Error S2801: VD CL033: Connect failed to localhost:1111 = localhost:1111. at line 1 of Top-Level: backup '/dev/null'

[HughWilliams]

The gdb back trace "??" entries would indicate the Virtuoso binary (virtuoso-t) is stripped of symbols, so you need to build and unstripped debug binary with the symbols in place as follows:

1. Edit ~/Makefile

2. To CONFIGURE_ARGS add --with-debug

3. To CONFIGURE_ENV prepend CC='cc -g’

4. Then do "make clean all deinstall reinstall” to build a new debug unstripped binary (virtuoso-t)

5. Start database with this new binary and force the crash condition again to generate a new core file

6. Use gdb to load core file

   gdb virtuoso-t core

7. At the prompt, type "bt" or “backtrace” to backtrace through stack and provide the output when top of stack is reached.

Either way as the database is not starting it looks as if a crash dump and recovery is required as indicated previously:

http://docs.openlinksw.com/virtuoso/databaseadmsrv.html#backup

One other thing might be to rename the "virtuoso.trx" transaction log and try to start just in case the corruption is in the trx file itself ...

[BobTheta]

I created a new database by changing databasefile in virtuoso.ini. Then Virtuoso could startup. However, after I loaded ttl files into the new database for several hours, it broke down again. I tried to rename the virtuoso.trx as you indicated , then the database ran seccessfully.

I wonder the full disk may be the main reason...

[TallTed]

@BobTheta, @Miyayx -- Using the advice above and a current stable/7 or develop/7 build, has this issue been resolved for you, or does it persist? Please advise, so we can close this issue or take other appropriate next steps.