featurebase icon indicating copy to clipboard operation
featurebase copied to clipboard

GroupBy stucks and never return

Open young118 opened this issue 4 years ago • 3 comments

For bugs, please provide the following:

What's going wrong?

GroupBy stucks in some query, I didn't figure out the pattern. Let me describe my data and my query. I have a packet index with 5 fields: src_ip, dst_ip, src_mac ,dst_mac, dummy. They are all time field. When I do GroupBy(Rows(src_ip), Rows(dst_ip),Rows(src_mac), Rows(dst_mac), filter=Row(dummy=dummy, from=start_time, to=end_time)), if the filter query is empty, then the GroupBy just stucks, never return.And if I change the start_time and end_time to make the filter query not empty, the GroupBy returns the result.GroupBy(Rows(src_ip), Rows(dst_ip),Rows(src_mac), Rows(dst_mac)) also works normally.It seems that this has something to do with if filter query returns empty.

But, if I do GroupBy(Rows(src_ip), Rows(src_mac)) no matter if I add filter query or if filter query return empty, this works fine. So it seem that this also has something to do with the number of Rows() queries.

In another situation I met before, Changing the order of Rows() can solve the problem.For example,GroupBy(Rows(src_ip), Rows(src_mac)) stucks, but GroupBy(Rows(src_mac), Rows(src_ip)) works.

So, the problem seems to have something to do with the filter query result, number of Rows() and the order of Rows(). And I'm sure it is not slow, it just stucks, like falling into an infinite loop, because the data is very small, I have only 200 columns.And when it stucks, the CPU usage keeps high

What was expected?

Please tell me if it is a bug, or what I did wrong?

Steps to reproduce the behavior

Information about your environment (OS/architecture, CPU, RAM, cluster/solo, configuration, etc.)

ubuntu 18.04 4 core 8G solo default configuration I use docker to deploy pilosa, image: pilosa/pilosa:latest

For feature requests, please provide the following:

Description

Success criteria (What criteria will consider this ticket closeable?)

young118 avatar Sep 11 '20 12:09 young118

It seems a bug, I change latest to master, and this problem disappear.

young118 avatar Sep 18 '20 02:09 young118

The problem showed up again, I can not figure out the pattern, but it seems a infinite loop bug in GroupBy operation, with time field filter.

young118 avatar Oct 29 '20 06:10 young118

why i performe Rows() method, the error appear, following: "localhost:10101/index/user/query" -X POST -d 'Rows(age)' error: {"error":"parsing: parsing: \nparse error near IDENT (line 1 symbol 6 - line 1 symbol 9):\n"age"\n"}

curl "localhost:10101/index/user/query" -X POST -d 'Rows(field="age")' error: {"error":"executing: map reduce: unknown call: Rows"}

hwo can i use it correctly? please help

Danny19580907 avatar Jul 05 '21 11:07 Danny19580907