Query hangs WaitingQueue timeout [BUG]

[phact]

Describe the bug A clear and concise description of what the bug is.

Query hangs with repeated warning in the RAL logs: 2021-05-06 21:43:13.868|0|warning||||WaitingQueue 6_6 wait_until_num_bytes timed out num_bytes wanted: 400000000 total_bytes: 0|59236||||

Steps/Code to reproduce bug Please provide the simplest and most complete steps or code that will allow us to reproduce the bug. These should be: Minimal – Use as little code as possible that still produces the same problem Complete – Provide all parts needed to reproduce the problem Verifiable – Test the code you’re about to provide to make sure it reproduces the problem

It was consistently reproducing for a while but then started completing in 9 seconds.

from blazingsql import BlazingContext

bc = BlazingContext(initial_pool_size=1.0486e+10, enable_progress_bar=True)
#bc = BlazingContext(initial_pool_size=1.0486e+10)
#config_options = {'ENABLE_TASK_LOGS':True, 'ENABLE_OTHER_ENGINE_LOGS': True, 'BLAZING_LOGGING_DIRECTORY':'/logging_directory'}
# logging directory defaults to $CONDA_PREFIX/blazing_log
#bc = BlazingContext(......, config_options = config_options)


bc.create_table('all_files', ["/home/ubuntu/data/2010/02/data.parquet",
"/home/ubuntu/data/2010/12/data.parquet",
"/home/ubuntu/data/2010/10/data.parquet",
"/home/ubuntu/data/2010/06/data.parquet",
"/home/ubuntu/data/2010/09/data.parquet",
"/home/ubuntu/data/2010/04/data.parquet",
"/home/ubuntu/data/2010/11/data.parquet",
"/home/ubuntu/data/2010/08/data.parquet",
"/home/ubuntu/data/2010/07/data.parquet",
"/home/ubuntu/data/2010/01/data.parquet",
"/home/ubuntu/data/2010/05/data.parquet",
"/home/ubuntu/data/2011/02/data.parquet",
"/home/ubuntu/data/2011/12/data.parquet",
"/home/ubuntu/data/2011/10/data.parquet",
"/home/ubuntu/data/2011/06/data.parquet",
"/home/ubuntu/data/2011/09/data.parquet",
"/home/ubuntu/data/2011/03/data.parquet",
"/home/ubuntu/data/2011/04/data.parquet",
"/home/ubuntu/data/2011/11/data.parquet",
"/home/ubuntu/data/2011/08/data.parquet",
"/home/ubuntu/data/2011/07/data.parquet",
"/home/ubuntu/data/2011/01/data.parquet",
"/home/ubuntu/data/2011/05/data.parquet",
"/home/ubuntu/data/2009/02/data.parquet",
"/home/ubuntu/data/2009/12/data.parquet",
"/home/ubuntu/data/2009/10/data.parquet",
"/home/ubuntu/data/2009/06/data.parquet",
"/home/ubuntu/data/2009/09/data.parquet",
"/home/ubuntu/data/2009/03/data.parquet",
"/home/ubuntu/data/2009/04/data.parquet",
"/home/ubuntu/data/2009/11/data.parquet",
"/home/ubuntu/data/2009/08/data.parquet",
"/home/ubuntu/data/2009/07/data.parquet",
"/home/ubuntu/data/2009/01/data.parquet",
"/home/ubuntu/data/2009/05/data.parquet",
"/home/ubuntu/data/2012/02/data.parquet",
"/home/ubuntu/data/2012/03/data.parquet",
"/home/ubuntu/data/2012/04/data.parquet",
"/home/ubuntu/data/2012/01/data.parquet",
"/home/ubuntu/data/2012/05/data.parquet"])

bc.describe_table("all_files")

result = bc.sql("select avg(pickup_latitude-dropoff_latitude), avg(tip_amount),  vendor_id from all_files group by vendor_id")
print(result)

Expected behavior A clear and concise description of what you expected to happen.

Query results in about 10 seconds when things are working normally.

Environment overview (please complete the following information)

• Environment location: [Bare-metal, Docker, Cloud(specify cloud provider)] AWS

• Method of BlazingSQL install: [conda, Docker, or from source] Conda

  • If method of install is [Docker], provide docker pull & docker run commands used

• BlazingSQL Version which can be obtained by doing as follows:

import blazingsql print(blazingsql.info()) BlazingSQL version (git hash): ff4ece0366a4d76bf533baeb03dd03bdfc5232be BlazingSQL branch name: HEAD BlazingSQL branch tag: v0.19.0 BlazingSQL build id: 0 BlazingSQL compiler version: GNU /usr/bin/c++ 7.5.0 BlazingSQL cuda flags: -Xcompiler -Wno-parentheses -gencode=arch=compute_60,code=sm_60 -gencode=arch=compute_70,code=sm_70 -gencode=arch=compute_75,code=sm_75 -gencode=arch=compute_75,code=compute_75 --expt-extended-lambda --expt-relaxed-constexpr -Werror=cross-execution-space-call -Xcompiler -Wall,-Wno-error=deprecated-declarations --default-stream=per-thread -DHT_DEFAULT_ALLOCATOR BlazingSQL Operating system kernel: Linux-5.4.0-1038-aws BlazingSQL Operating system architecture: x86_64 BlazingSQL Linux Operating system release: NAME=Ubuntu|VERSION=16.04.7 LTS (Xenial Xerus)|ID=ubuntu|ID_LIKE=debian|PRETTY_NAME=Ubuntu 16.04.7 LTS|VERSION_ID=16.04|HOME_URL=http://www.ubuntu.com/|SUPPORT_URL=http://help.ubuntu.com/|BUG_REPORT_URL=http://bugs.launchpad.net/ubuntu/|VERSION_CODENAME=xenial|UBUNTU_CODENAME=xenial None

**Environment details**
Please run and paste the output of the `print_env.sh` script here, to gather any other relevant environment details

**Additional context**
Add any other context about the problem here.

**----For BlazingSQL Developers----**
**Suspected source of the issue**
Where and what are potential sources of the issue

**Other design considerations**
What components of the engine could be affected by this?