How did we implement parallel reading by threads in 0.8 new version?

[jackjoesh]

Do we support parallel read by threads? I can't see the parallel_read_threads in the settings of new version. And I have found this issue: https://github.com/datafuselabs/databend/pull/4480 How did we implemented parallel reading by threads now?

[bohutang]

Forward @dantengsky to give more information about this question :)

[dantengsky]

the setting of max_threads is used instead

https://github.com/datafuselabs/databend/blob/97c51b5ea3d1c1e49b154c8d80307e408bd4c12f/src/query/storages/fuse/src/operations/read.rs#L129-L148

we can tweak this setting at runtime, for example

set max_threads=1;
select ... from ...;

[jackjoesh]

the setting of max_threads is used instead

https://github.com/datafuselabs/databend/blob/97c51b5ea3d1c1e49b154c8d80307e408bd4c12f/src/query/storages/fuse/src/operations/read.rs#L129-L148

we can tweak this setting at runtime, for example

set max_threads=1;
select ... from ...;

Thank you for your reply. In addition, I would like to ask. It was originally supported to set more than the number of queries and then go to distributed query. Is this parameter still available? Where can it be set?

[FANNG1]

we must set num_cpus in databend-query.toml not max_threads, this seems confusing for me.

        {
            // Set max threads.
            let cpus = if conf.query.num_cpus == 0 {
                num_cpus::get() as u64
            } else {
                conf.query.num_cpus
            };
            ret.set_max_threads(cpus)?;
        }