octosql icon indicating copy to clipboard operation
octosql copied to clipboard

Panic in parquet query

Open yusufozturk opened this issue 10 months ago • 0 comments

I was using ClickBench queries to test octosql performance.

I get following error for query number 23:

panic: runtime error: index out of range [1986948931] with length 4192

Query:

octosql.exe "SELECT * FROM hits.parquet WHERE URL LIKE '%google%' ORDER BY EventTime LIMIT 10"

Full error:

goroutine 1 [running]:
github.com/segmentio/parquet-go.(*byteArrayDictionary).Index(0x58dd85?, 0x11c340?)
        /home/runner/go/pkg/mod/github.com/cube2222/[email protected]/dictionary.go:86 +0xed
github.com/segmentio/parquet-go.(*indexedPageReader).ReadValues(0xc02da4df20, {0xc002901fe0, 0xaa, 0xb8045a?})
        /home/runner/go/pkg/mod/github.com/cube2222/[email protected]/dictionary.go:338 +0x89
github.com/segmentio/parquet-go.(*columnChunkReader).readValuesFromCurrentPage(0xc0028b80c0)
        /home/runner/go/pkg/mod/github.com/cube2222/[email protected]/column_chunk.go:135 +0x90
github.com/segmentio/parquet-go.(*columnChunkReader).readValues(0xfcf180?)
        /home/runner/go/pkg/mod/github.com/cube2222/[email protected]/column_chunk.go:115 +0x29
github.com/segmentio/parquet-go.columnReadRowFuncOfLeaf.func1({0xc02e69b710?, 0x2, 0x2}, 0x0?, {0xc0028b8000, 0x0?, 0x0?})
        /home/runner/go/pkg/mod/github.com/cube2222/[email protected]/column_chunk.go:326 +0xc5
github.com/segmentio/parquet-go.makeColumnReadRowFunc.func1({0x0?, 0x3?, 0x0?}, 0x0?, {0xc0028b8000, 0x69, 0x69})
        /home/runner/go/pkg/mod/github.com/cube2222/[email protected]/schema.go:163 +0xa3
github.com/segmentio/parquet-go.(*rowGroupRowReader).ReadRow(0x0?, {0x0?, 0x0, 0x0?})
        /home/runner/go/pkg/mod/github.com/cube2222/[email protected]/row_group.go:306 +0xb7
github.com/segmentio/parquet-go.(*reader).ReadRow(0xc00165c370, {0x0?, 0x0, 0x0?})
        /home/runner/go/pkg/mod/github.com/cube2222/[email protected]/reader.go:276 +0xb1
github.com/segmentio/parquet-go.(*Reader).ReadRow(0xc00165c360, {0x0, 0x0, 0x0})
        /home/runner/go/pkg/mod/github.com/cube2222/[email protected]/reader.go:221 +0x65
github.com/cube2222/octosql/datasources/parquet.(*DatasourceExecuting).Run(0xc00142f140, {{0x115d8f0?, 0xc000384e40?}, 0x0?}, 0xc00142f1d0, 0x0?)
        /home/runner/work/octosql/octosql/datasources/parquet/execution.go:47 +0x512
github.com/cube2222/octosql/execution/nodes.(*Filter).Run(0xc0003dac20, {{0x115d8f0?, 0xc000384e40?}, 0x0?}, 0xc0000e15e0, 0x1149976?)
        /home/runner/work/octosql/octosql/execution/nodes/filter.go:23 +0xfc
github.com/cube2222/octosql/outputs/batch.(*OutputPrinter).Run(0xc00165c2d0, {{0x115d8f0?, 0xc000384e40?}, 0x0?})
        /home/runner/work/octosql/octosql/outputs/batch/live_output.go:116 +0x4b9
github.com/cube2222/octosql/cmd.glob..func4(0x1799140, {0xc00037ead0, 0x1, 0x1?})
        /home/runner/work/octosql/octosql/cmd/root.go:458 +0x3b34
github.com/spf13/cobra.(*Command).execute(0x1799140, {0xc00009e3b0, 0x1, 0x1})
        /home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:856 +0x67c
github.com/spf13/cobra.(*Command).ExecuteC(0x1799140)
        /home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:974 +0x3b4
github.com/spf13/cobra.(*Command).Execute(...)
        /home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:902
github.com/spf13/cobra.(*Command).ExecuteContext(...)
        /home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:895
github.com/cube2222/octosql/cmd.Execute({0x115d848?, 0xc00022edc0?})
        /home/runner/work/octosql/octosql/cmd/root.go:471 +0x53
main.main()
        /home/runner/work/octosql/octosql/main.go:24 +0xe8

Same query runs on DuckDB:

image

yusufozturk avatar Aug 22 '23 21:08 yusufozturk