Crash when using LEVENSHTEIN()

[AlexeyRemenyak]

To Reproduce
Steps to reproduce the behavior:

1. create table test (f1 text);
2. insert into test (f1) values ('(ТЕРРИТОРИАЛЬНОЙ) ГЖД - ФИЛИАЛА ОАО "РЖД" МЕЖРЕГИОНАЛЬНОЙ "НА СТ. ГОРЬКИЙ ОАО "РЖД"');
3. select *, LEVENSHTEIN('РЖД', f1) from test where match('РЖД');

Describe the environment:

1:

• Manticore 4.2.1 ebf51657a@220217 release (columnar 1.14.1 fce43f7@220203)
• Linux 5.10.0-10-amd64 SMP Debian 5.10.84-1 (2021-12-08) x86_64 GNU/Linux

2:

• Manticore 4.2.0 15e927b28@211223 release (columnar 1.11.4 327b3d4@211223)
• Linux 4.18.0-348.7.1.el8_5.x86_64 SMP Tue Dec 21 09:11:23 PST 2021 x86_64 x86_64 x86_64 GNU/Linux

Messages from log files:
Log from EL8 version:

------- FATAL: CRASH DUMP -------
[Fri Feb 18 13:55:29.208 2022] [3014832]

--- crashed SphinxQL request dump ---
select *, LEVENSHTEIN('РЖД', f1) from test where match('РЖД')
--- request dump end ---
--- local index:test
Manticore 4.2.0 15e927b28@211223 release
Handling signal 11
-------------- backtrace begins here ---------------
Program compiled with Clang 11.0.0
Configured with flags: Configured with these definitions: -DDISTR_BUILD=rhel8 -DUSE_SYSLOG=1 -DWITH_GALERA=1 -DWITH_RE2=1 -DWITH_RE2_FORCE_STATIC=1 -DWITH_STEMMER=1 -DWITH_STEMMER_FORCE_STATIC=1 -DWITH_ICU=1 -DWITH_ICU_FORCE_STATIC=1 -DWITH_SSL=1 -DWITH_ZLIB=1 -DWITH_ODBC=1 -DDL_ODBC=1 -DODBC_LIB=libodbc.so.2 -DWITH_EXPAT=1 -DDL_EXPAT=1 -DEXPAT_LIB=libexpat.so.1 -DWITH_ICONV=1 -DWITH_MYSQL=1 -DDL_MYSQL=1 -DMYSQL_LIB=libmariadb.so.3 -DWITH_POSTGRESQL=1 -DDL_POSTGRESQL=1 -DPOSTGRESQL_LIB=libpq.so.5 -DLOCALDATADIR=/var/lib/manticore/data -DFULL_SHARE_DIR=/usr/share/manticore
Host OS is Linux x86_64
Stack bottom = 0x7f27fc023ea0, thread stack size = 0x20000
Trying manual backtrace:
Something wrong with thread stack, manual backtrace may be incorrect (fp=0x20000)
Wrong stack limit or frame pointer, manual backtrace failed (fp=0x20000, stack=0x7f27fc020000, stacksize=0x20000)
Trying system backtrace:
[Fri Feb 18 13:55:29.648 2022] [3014831] watchdog: main process 3014832 killed dirtily with signal 11, core dumped, will be restarted
[Fri Feb 18 13:55:29.649 2022] [3014831] watchdog: main process 3015034 forked ok
[Fri Feb 18 13:55:29.649 2022] [3015034] starting daemon version '4.2.0 15e927b28@211223 release (columnar 1.11.4 327b3d4@211223)' ...
[Fri Feb 18 13:55:29.649 2022] [3015034] listening on 127.0.0.1:9312 for sphinx and http(s)
[Fri Feb 18 13:55:29.649 2022] [3015034] listening on 127.0.0.1:9306 for mysql
[Fri Feb 18 13:55:29.649 2022] [3015034] listening on 127.0.0.1:9308 for sphinx and http(s)
[Fri Feb 18 13:55:29.661 2022] [3015038] binlog: replaying log /var/lib/manticore/binlog/binlog.001

--- crashed invalid query ---

--- request dump end ---
--- local index:
Manticore 4.2.0 15e927b28@211223 release
Handling signal 11
-------------- backtrace begins here ---------------
Program compiled with Clang 11.0.0
Configured with flags: Configured with these definitions: -DDISTR_BUILD=rhel8 -DUSE_SYSLOG=1 -DWITH_GALERA=1 -DWITH_RE2=1 -DWITH_RE2_FORCE_STATIC=1 -DWITH_STEMMER=1 -DWITH_STEMMER_FORCE_STATIC=1 -DWITH_ICU=1 -DWITH_ICU_FORCE_STATIC=1 -DWITH_SSL=1 -DWITH_ZLIB=1 -DWITH_ODBC=1 -DDL_ODBC=1 -DODBC_LIB=libodbc.so.2 -DWITH_EXPAT=1 -DDL_EXPAT=1 -DEXPAT_LIB=libexpat.so.1 -DWITH_ICONV=1 -DWITH_MYSQL=1 -DDL_MYSQL=1 -DMYSQL_LIB=libmariadb.so.3 -DWITH_POSTGRESQL=1 -DDL_POSTGRESQL=1 -DPOSTGRESQL_LIB=libpq.so.5 -DLOCALDATADIR=/var/lib/manticore/data -DFULL_SHARE_DIR=/usr/share/manticore
Host OS is Linux x86_64
Stack bottom = 0x2805670, thread stack size = 0x20000
Trying manual backtrace:
Something wrong with thread stack, manual backtrace may be incorrect (fp=0x20000)
Wrong stack limit or frame pointer, manual backtrace failed (fp=0x20000, stack=0x2810000, stacksize=0x20000)
Trying system backtrace:
begin of system symbols:
searchd[0x5a7870]
searchd[0x47a02f]
/lib64/libpthread.so.0( 0x12c30)[0x7f2837ef1c30]
searchd[0xc72054]
searchd[0xc70bc9]
searchd[0xc7058b]
searchd[0xc72a33]
searchd[0x4daff5]
searchd[0xd5d67c]
searchd[0xd7756f]
-------------- backtrace ends here ---------------

Additional context
query select * from test where match('РЖД'); does not cause a crash

just like the query below

create table test (f1 text);  
insert into test (f1) values ('(ТЕРРИТОРИАЛЬНОЙ) ГЖД - ОАО "РЖД" МЕЖРЕГИОНАЛЬНОЙ "НА СТ. "РЖД"');  
select *, LEVENSHTEIN('РЖД', f1) from test where match('РЖД');  
 --------------------- --------------------------------------------------------------------------------------------------------------- ---------------------------   
| id                  | f1                                                                                                            | levenshtein('РЖД', f1)    |  
 --------------------- --------------------------------------------------------------------------------------------------------------- ---------------------------   
| 6774898862139113473 | (ТЕРРИТОРИАЛЬНОЙ) ГЖД - ОАО "РЖД" МЕЖРЕГИОНАЛЬНОЙ "НА СТ. "РЖД"                                               |                       103 |  
 --------------------- --------------------------------------------------------------------------------------------------------------- ---------------------------   
1 row in set (0.000 sec)  

[sanikolaev]

Thank you for such a clear minimal reproducible example. I wish all crash reports were like this.

[githubmanticore]

➤ Ilya Kuznetsov commented:

Already fixed in latest master (43a9fef4f310ae39cc1fcbf8a72ec1316ac8be20). Crash occurred because f1 is a stored field and you can't use expression on stored fields, only on attributes.