manticoresearch
manticoresearch copied to clipboard
SphinxSE - Misc Errors keep appearing periodically.
Describe the bug
Many queries being sent through MariaDB 10 SphinxSE, Intermittently errors are being returned.
Most of the time the query succeeds.
Otherwise.
Error:mysqli::query(): (HY000/1430): There was a problem processing the query on the foreign data source. Data source error: searchd error: index realtime_media_123: syntax error, unexpecte
Error:mysqli::query(): (HY000/1430): There was a problem processing the query on the foreign data source. Data source error: search query already specified; 's' is redundant
Error:mysqli::query(): (HY000/1430): There was a problem processing the query on the foreign data source. Data source error: search query already specified; 'f' is redundant
Error:mysqli::query(): (HY000/1430): There was a problem processing the query on the foreign data source. Data source error: search query already specified; 'n' is redundant
Error:mysqli::query(): (HY000/1430): There was a problem processing the query on the foreign data source. Data source error: search query already specified; '0' is redundant
Error:mysqli::query(): (HY000/1430): There was a problem processing the query on the foreign data source. Data source error: search query already specified; ' ' is redundant
Error:mysqli::query(): (HY000/1430): There was a problem processing the query on the foreign data source. Data source error: search query already specified; 'd' is redundant
Error:mysqli::query(): (HY000/1430): There was a problem processing the query on the foreign data source. Data source error: search query already specified; ' ' is redundant
Error:mysqli::query(): (HY000/1430): There was a problem processing the query on the foreign data source. Data source error: search query already specified; ' ' is redundant
Error:mysqli::query(): (HY000/1430): There was a problem processing the query on the foreign data source. Data source error: search query already specified; 's' is redundant
Error:mysqli::query(): (HY000/1430): There was a problem processing the query on the foreign data source. Data source error: search query already specified; ' ' is redundant
Error:mysqli::query(): (HY000/1430): There was a problem processing the query on the foreign data source. Data source error: search query already specified; '3' is redundant
Error:mysqli::query(): (HY000/1430): There was a problem processing the query on the foreign data source. Data source error: search query already specified; 'h' is redundant
Error:mysqli::query(): (HY000/1430): There was a problem processing the query on the foreign data source. Data source error: search query already specified; '2' is redundant
Error:mysqli::query(): (HY000/1430): There was a problem processing the query on the foreign data source. Data source error: searchd error: index realtime_media_123: syntax error, unexpecte
Error:mysqli::query(): (HY000/1430): There was a problem processing the query on the foreign data source. Data source error: searchd error: index realtime_media_123: syntax error, unexpecte
Error:mysqli::query(): (HY000/1430): There was a problem processing the query on the foreign data source. Data source error: search query already specified; '8' is redundant
Error:mysqli::query(): (HY000/1430): There was a problem processing the query on the foreign data source. Data source error: search query already specified; '0' is redundant
Error:mysqli::query(): (HY000/1430): There was a problem processing the query on the foreign data source. Data source error: search query already specified; 'i' is redundant
Error:mysqli::query(): (HY000/1430): There was a problem processing the query on the foreign data source. Data source error: searchd error: index realtime_media_123: syntax error, unexpecte
Error:mysqli::query(): (HY000/1430): There was a problem processing the query on the foreign data source. Data source error: searchd error: index realtime_media_123: syntax error, unexpecte
Error:mysqli::query(): (HY000/1430): There was a problem processing the query on the foreign data source. Data source error: search query already specified; 't' is redundant
Error:mysqli::query(): (HY000/1430): There was a problem processing the query on the foreign data source. Data source error: search query already specified; 'e' is redundant
Error:mysqli::query(): (HY000/1430): There was a problem processing the query on the foreign data source. Data source error: search query already specified; ' ' is redundant
Error:mysqli::query(): (HY000/1430): There was a problem processing the query on the foreign data source. Data source error: search query already specified; 't' is redundant
Error:mysqli::query(): (HY000/1430): There was a problem processing the query on the foreign data source. Data source error: searchd error: index realtime_media_123: syntax error, unexpecte
Error:mysqli::query(): (HY000/1430): There was a problem processing the query on the foreign data source. Data source error: searchd error: index realtime_media_123: syntax error, unexpecte
Error:mysqli::query(): (HY000/1430): There was a problem processing the query on the foreign data source. Data source error: search query already specified; 'n' is redundant
Error:mysqli::query(): (HY000/1430): There was a problem processing the query on the foreign data source. Data source error: searchd error: index realtime_media_123: syntax error, unexpecte
Error:mysqli::query(): (HY000/1430): There was a problem processing the query on the foreign data source. Data source error: searchd error: index realtime_media_123: syntax error, unexpecte
To Reproduce
Steps to reproduce the behavior:
- SphinxSE Storage engine
Sample queries
(sphinxmedia.query='mode=extended2;index=realtime_media_123;@indexedTags en;filter=moderationstatus,1;filter=vhost,123;filter=parentgroup,422841;range=channelleft,302,373;filter=status,3;filter=context,1;filter=hidden,0;filter=filetype,1;sort=extended:upload DESC,upload DESC;limit=16;offset=16;maxmatches=1000000;')
Expected behavior
A clear and concise description of what you expected to happen.
Describe the environment:
- Manticore Search 3.6
- Linux ec2-db-slave2 5.8.0-1041-aws #43~20.04.1-Ubuntu SMP Thu Jul 15 11:07:29 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
- MariaDB 10.6
Messages from log files:
Messages from searchd.log and query.log (if applicable).
Additional context
CREATE TABLE `sphinxmedia` (
`id` bigint(10) unsigned NOT NULL,
`weight` int(11) NOT NULL,
`query` varchar(1000) NOT NULL,
`_sph_geodist` float DEFAULT NULL,
KEY `query` (`query`)
) ENGINE=SPHINX DEFAULT CHARSET=utf8mb3 CONNECTION='sphinx://172.30.3.35:9312'
172.30.3.35 is an HaProxy server load balancing the queries
Indextool runs fine on the indexes.
I've tried isolating it to a specific searchd but all of them are doing the same.
Index conf
index realtime_media_123
{
rt_mem_limit = 2048M
type = rt
dict = keywords
path = /var/lib/manticore/data/realtimemedia_123
rt_field = indexedtitle
rt_field = indexedmessage
rt_field = indexedtags
rt_field = indexedfilename
rt_field = indexedfirstname
rt_field = indexedlastname
rt_field = indexeduser
rt_attr_json = indexedmetadata
rt_attr_string = filename
rt_attr_string = title
rt_attr_uint = filetype
rt_attr_uint = status
rt_attr_uint = channel
rt_attr_uint = channelleft
rt_attr_uint = vhost
rt_attr_uint = moderationstatus
rt_attr_uint = context
rt_attr_uint = parentid
rt_attr_uint = parentuserid
rt_attr_uint = uid
rt_attr_uint = privacy
rt_attr_uint = commentcount
rt_attr_uint = approvedcomments
rt_attr_uint = notdeniedcomments
rt_attr_uint = hits
rt_attr_uint = filesize
rt_attr_uint = ofilesize
rt_attr_uint = width
rt_attr_uint = height
rt_attr_uint = length
rt_attr_uint = offensive
rt_attr_uint = votecount
rt_attr_uint = rotation
rt_attr_uint = autoblog
rt_attr_uint = hidden
rt_attr_uint = children
rt_attr_timestamp = upload
rt_attr_timestamp = date
rt_attr_timestamp = converttime
rt_attr_timestamp = lastupdatetime
rt_attr_timestamp = startdate
rt_attr_timestamp = enddate
rt_attr_float = latituderadians
rt_attr_float = longituderadians
rt_attr_float = rating
rt_attr_float = conversiontime
rt_attr_multi = parentchannels
rt_attr_multi = gid
rt_attr_multi = parentgroup
rt_attr_multi = miscsort
min_infix_len = 4
stored_fields = indexedtitle,indexedmessage,indexedtags,indexedfilename,indexedfirstname,indexedlastname,indexeduser
}
searchd
searchd
{
workers=threads
log = /var/log/manticore/searchd.log
binlog_path = /var/lib/manticore/binlog
query_log = /var/log/manticore/query.log
persistent_connections_limit = 1
network_timeout = 5
client_timeout = 100
rt_flush_period = 7200
binlog_max_log_size = 128M
pid_file = /var/run/manticore/searchd.pid
seamless_rotate = 1
preopen_indexes = 0
unlink_old = 1
attr_flush_period = 900
max_packet_size = 8M
max_filters = 256
max_filter_values = 4096
listen_backlog = 1000
listen = 0.0.0.0:9312:sphinx
listen = 0.0.0.0:9306:mysql
listen = 0.0.0.0:9308:http
}
Definitely seems related to MariaDB 10.6.. Reverted my servers back to 10.5 and the errors have disappeared