mongo icon indicating copy to clipboard operation
mongo copied to clipboard

Published 20 hours ago • verified publisher icon mongodb

SERVER-79010: after we reopen wiredtiger_open, we shold close the old connection

Open y123456yz opened this issue 11 months ago • 7 comments

after we reopen wiredtiger_open, we shold close the old connection

if we not close the old open wiredtiger, There will be a Resource leak.

y123456yz avatar Jul 17 '23 12:07 y123456yz

https://jira.mongodb.org/browse/SERVER-79010

y123456yz avatar Jul 18 '23 03:07 y123456yz

Hi @y123456yz thanks for the contribution. I tested this locally, but I got a segmentation fault in the new code.

This is what I did:

  • Start 4.2
  • Shutdown 4.2 cleanly
  • Start up with the patched binary on the same data directory
{"t":{"$date":"2023-11-02T17:57:38.966+00:00"},"s":"I",  "c":"CONTROL",  "id":4615611, "ctx":"initandlisten","msg":"MongoDB starting","attr":{"pid":3185758,"port":27017,"dbPath":"/home/ubuntu/downloads/4.2","architecture":"64-bit","host":"ip-10-122-7-78"}}
{"t":{"$date":"2023-11-02T17:57:38.966+00:00"},"s":"I",  "c":"CONTROL",  "id":20533,   "ctx":"initandlisten","msg":"DEBUG build (which is slower)"}
{"t":{"$date":"2023-11-02T17:57:38.967+00:00"},"s":"I",  "c":"CONTROL",  "id":23403,   "ctx":"initandlisten","msg":"Build Info","attr":{"buildInfo":{"version":"7.2.0-alpha0","gitVersion":"unknown","openSSLVersion":"OpenSSL 3.0.2 15 Mar 2022","modules":[],"allocator":"tcmalloc","environment":{"distarch":"aarch64","target_arch":"aarch64"}}}}
{"t":{"$date":"2023-11-02T17:57:38.967+00:00"},"s":"I",  "c":"CONTROL",  "id":51765,   "ctx":"initandlisten","msg":"Operating System","attr":{"os":{"name":"Ubuntu","version":"22.04"}}}
{"t":{"$date":"2023-11-02T17:57:38.967+00:00"},"s":"I",  "c":"CONTROL",  "id":21951,   "ctx":"initandlisten","msg":"Options set by command line","attr":{"options":{"storage":{"dbPath":"/home/ubuntu/downloads/4.2/"}}}}
[New Thread 0xffffb09ae340 (LWP 3185814)]
[New Thread 0xffffb019e340 (LWP 3185815)]
[New Thread 0xffffaf98e340 (LWP 3185816)]
{"t":{"$date":"2023-11-02T17:57:38.994+00:00"},"s":"I",  "c":"STORAGE",  "id":22270,   "ctx":"initandlisten","msg":"Storage engine to use detected by data files","attr":{"dbpath":"/home/ubuntu/downloads/4.2","storageEngine":"wiredTiger"}}
[New Thread 0xffffaf07e340 (LWP 3185817)]
{"t":{"$date":"2023-11-02T17:57:39.001+00:00"},"s":"I",  "c":"STORAGE",  "id":22315,   "ctx":"initandlisten","msg":"Opening WiredTiger","attr":{"config":"create,cache_size=15174M,session_max=33000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,remove=true,path=journal,compressor=snappy),builtin_extension_config=(zstd=(compression_level=6)),file_manager=(close_idle_time=600,close_scan_interval=10,close_handle_minimum=2000),statistics_log=(wait=0),json_output=(error,message),verbose=[recovery_progress:1,checkpoint_progress:1,compact_progress:1,backup:0,checkpoint:0,compact:0,evict:0,history_store:0,recovery:0,rts:0,salvage:0,tiered:0,timestamp:0,transaction:0,verify:0,log:0],debug_mode=(corruption_abort=false,),"}}

Thread 1 "mongod" received signal SIGSEGV, Segmentation fault.
0x0000ffffc4372eb0 in mongo::WiredTigerKVEngine::_openWiredTiger (this=0xffffb2260620, path="/home/ubuntu/downloads/4.2", wtOpenConfig="create,cache_size=15174M,session_max=33000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,remove=true,path=journal,compressor=snappy),builtin_extension_co"...) at src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp:657
657         invariantWTOK(_conn->close(_conn, nullptr), nullptr);
(gdb) p _conn
$1 = (WT_CONNECTION *) 0x0
(gdb)

When wiredtiger_open fails, the WT_CONNECTION object remains null and closed. For this reason, I don't think it is safe to make this change. Please let me know if you agree.

louiswilliams avatar Nov 02 '23 18:11 louiswilliams

Hi @y123456yz thanks for the contribution. I tested this locally, but I got a segmentation fault in the new code.

This is what I did:

  • Start 4.2
  • Shutdown 4.2 cleanly
  • Start up with the patched binary on the same data directory
{"t":{"$date":"2023-11-02T17:57:38.966+00:00"},"s":"I",  "c":"CONTROL",  "id":4615611, "ctx":"initandlisten","msg":"MongoDB starting","attr":{"pid":3185758,"port":27017,"dbPath":"/home/ubuntu/downloads/4.2","architecture":"64-bit","host":"ip-10-122-7-78"}}
{"t":{"$date":"2023-11-02T17:57:38.966+00:00"},"s":"I",  "c":"CONTROL",  "id":20533,   "ctx":"initandlisten","msg":"DEBUG build (which is slower)"}
{"t":{"$date":"2023-11-02T17:57:38.967+00:00"},"s":"I",  "c":"CONTROL",  "id":23403,   "ctx":"initandlisten","msg":"Build Info","attr":{"buildInfo":{"version":"7.2.0-alpha0","gitVersion":"unknown","openSSLVersion":"OpenSSL 3.0.2 15 Mar 2022","modules":[],"allocator":"tcmalloc","environment":{"distarch":"aarch64","target_arch":"aarch64"}}}}
{"t":{"$date":"2023-11-02T17:57:38.967+00:00"},"s":"I",  "c":"CONTROL",  "id":51765,   "ctx":"initandlisten","msg":"Operating System","attr":{"os":{"name":"Ubuntu","version":"22.04"}}}
{"t":{"$date":"2023-11-02T17:57:38.967+00:00"},"s":"I",  "c":"CONTROL",  "id":21951,   "ctx":"initandlisten","msg":"Options set by command line","attr":{"options":{"storage":{"dbPath":"/home/ubuntu/downloads/4.2/"}}}}
[New Thread 0xffffb09ae340 (LWP 3185814)]
[New Thread 0xffffb019e340 (LWP 3185815)]
[New Thread 0xffffaf98e340 (LWP 3185816)]
{"t":{"$date":"2023-11-02T17:57:38.994+00:00"},"s":"I",  "c":"STORAGE",  "id":22270,   "ctx":"initandlisten","msg":"Storage engine to use detected by data files","attr":{"dbpath":"/home/ubuntu/downloads/4.2","storageEngine":"wiredTiger"}}
[New Thread 0xffffaf07e340 (LWP 3185817)]
{"t":{"$date":"2023-11-02T17:57:39.001+00:00"},"s":"I",  "c":"STORAGE",  "id":22315,   "ctx":"initandlisten","msg":"Opening WiredTiger","attr":{"config":"create,cache_size=15174M,session_max=33000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,remove=true,path=journal,compressor=snappy),builtin_extension_config=(zstd=(compression_level=6)),file_manager=(close_idle_time=600,close_scan_interval=10,close_handle_minimum=2000),statistics_log=(wait=0),json_output=(error,message),verbose=[recovery_progress:1,checkpoint_progress:1,compact_progress:1,backup:0,checkpoint:0,compact:0,evict:0,history_store:0,recovery:0,rts:0,salvage:0,tiered:0,timestamp:0,transaction:0,verify:0,log:0],debug_mode=(corruption_abort=false,),"}}

Thread 1 "mongod" received signal SIGSEGV, Segmentation fault.
0x0000ffffc4372eb0 in mongo::WiredTigerKVEngine::_openWiredTiger (this=0xffffb2260620, path="/home/ubuntu/downloads/4.2", wtOpenConfig="create,cache_size=15174M,session_max=33000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,remove=true,path=journal,compressor=snappy),builtin_extension_co"...) at src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp:657
657         invariantWTOK(_conn->close(_conn, nullptr), nullptr);
(gdb) p _conn
$1 = (WT_CONNECTION *) 0x0
(gdb)

When wiredtiger_open fails, the WT_CONNECTION object remains null and closed. For this reason, I don't think it is safe to make this change. Please let me know if you agree.

hi， @louiswilliams What version are you using 4.2.x？

I did the test with the 5.0.13 code as you said, it's ok, No problem. I compiled the wiredtiger code, did a demo test, and it was fine。

thanks.

y123456yz avatar Nov 03 '23 02:11 y123456yz

Hi @y123456yz thanks for the contribution. I tested this locally, but I got a segmentation fault in the new code.

This is what I did:

  • Start 4.2
  • Shutdown 4.2 cleanly
  • Start up with the patched binary on the same data directory
{"t":{"$date":"2023-11-02T17:57:38.966+00:00"},"s":"I",  "c":"CONTROL",  "id":4615611, "ctx":"initandlisten","msg":"MongoDB starting","attr":{"pid":3185758,"port":27017,"dbPath":"/home/ubuntu/downloads/4.2","architecture":"64-bit","host":"ip-10-122-7-78"}}
{"t":{"$date":"2023-11-02T17:57:38.966+00:00"},"s":"I",  "c":"CONTROL",  "id":20533,   "ctx":"initandlisten","msg":"DEBUG build (which is slower)"}
{"t":{"$date":"2023-11-02T17:57:38.967+00:00"},"s":"I",  "c":"CONTROL",  "id":23403,   "ctx":"initandlisten","msg":"Build Info","attr":{"buildInfo":{"version":"7.2.0-alpha0","gitVersion":"unknown","openSSLVersion":"OpenSSL 3.0.2 15 Mar 2022","modules":[],"allocator":"tcmalloc","environment":{"distarch":"aarch64","target_arch":"aarch64"}}}}
{"t":{"$date":"2023-11-02T17:57:38.967+00:00"},"s":"I",  "c":"CONTROL",  "id":51765,   "ctx":"initandlisten","msg":"Operating System","attr":{"os":{"name":"Ubuntu","version":"22.04"}}}
{"t":{"$date":"2023-11-02T17:57:38.967+00:00"},"s":"I",  "c":"CONTROL",  "id":21951,   "ctx":"initandlisten","msg":"Options set by command line","attr":{"options":{"storage":{"dbPath":"/home/ubuntu/downloads/4.2/"}}}}
[New Thread 0xffffb09ae340 (LWP 3185814)]
[New Thread 0xffffb019e340 (LWP 3185815)]
[New Thread 0xffffaf98e340 (LWP 3185816)]
{"t":{"$date":"2023-11-02T17:57:38.994+00:00"},"s":"I",  "c":"STORAGE",  "id":22270,   "ctx":"initandlisten","msg":"Storage engine to use detected by data files","attr":{"dbpath":"/home/ubuntu/downloads/4.2","storageEngine":"wiredTiger"}}
[New Thread 0xffffaf07e340 (LWP 3185817)]
{"t":{"$date":"2023-11-02T17:57:39.001+00:00"},"s":"I",  "c":"STORAGE",  "id":22315,   "ctx":"initandlisten","msg":"Opening WiredTiger","attr":{"config":"create,cache_size=15174M,session_max=33000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,remove=true,path=journal,compressor=snappy),builtin_extension_config=(zstd=(compression_level=6)),file_manager=(close_idle_time=600,close_scan_interval=10,close_handle_minimum=2000),statistics_log=(wait=0),json_output=(error,message),verbose=[recovery_progress:1,checkpoint_progress:1,compact_progress:1,backup:0,checkpoint:0,compact:0,evict:0,history_store:0,recovery:0,rts:0,salvage:0,tiered:0,timestamp:0,transaction:0,verify:0,log:0],debug_mode=(corruption_abort=false,),"}}

Thread 1 "mongod" received signal SIGSEGV, Segmentation fault.
0x0000ffffc4372eb0 in mongo::WiredTigerKVEngine::_openWiredTiger (this=0xffffb2260620, path="/home/ubuntu/downloads/4.2", wtOpenConfig="create,cache_size=15174M,session_max=33000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,remove=true,path=journal,compressor=snappy),builtin_extension_co"...) at src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp:657
657         invariantWTOK(_conn->close(_conn, nullptr), nullptr);
(gdb) p _conn
$1 = (WT_CONNECTION *) 0x0
(gdb)

When wiredtiger_open fails, the WT_CONNECTION object remains null and closed. For this reason, I don't think it is safe to make this change. Please let me know if you agree.

I merge the code to 4.2.24, Follow your test procedure， test is ok.

y123456yz avatar Nov 03 '23 05:11 y123456yz

hi, @louiswilliams

if there is a conclusion? thanks.

y123456yz avatar Nov 17 '23 03:11 y123456yz

I am not confident that this works correctly on versions 4.4+. Can you explain what motivated this change and your test setup?

louiswilliams avatar Nov 17 '23 14:11 louiswilliams

hi, @louiswilliams

the step as following:

  1. startup mongod(5.0.12, not contain this patch) /data/mongodb-5.0-new/mongo-r5.0.13/mongod --port 1122 --dbpath=./test/
  2. insert some data

use test switched to db test db.test.insert({xxxx:"ssssssssssss"}) WriteResult({ "nInserted" : 1 })

  1. shutdown db.shutdownServer()
  2. compile code with this patch, startup mongodd(5.0.12, contain this patch) ./mongod --port 1122 --dbpath=./test/

all the test is ok.

thanks.

y123456yz avatar Nov 20 '23 11:11 y123456yz