nvdbaranec

Remove support for skip_rows / num_rows options in the parquet reader.

1

Removes support for skip_rows / num_rows options in the parquet reader. Users retail control of what gets read via row groups.

[FEA] Resolve what to do with semi-redundant fields in cuIO table_metadata

6

This PR (https://github.com/rapidsai/cudf/pull/6318) adds a new field to the `table_metadata` struct, `schema_info`, which contains the column names for the entire hierarchy of returned columns, not just the root columns. This...

The casting code in the plugin is currently a collection of expensive string manipulation functionality (regex, etc). It exists to handle more exotic edge cases coming from CSV and JSON...

[FEA] Performance issue with the Parquet reader for very large schemas (especially when containing strings)

2

For parquet files that contain very large schemas with strings (either large numbers of columns, or large numbers of nested columns) we pay a very heavy price postprocessing the string...

There's only a very basic row group selection test in the C++ gtests. It would probably be useful to have a more thorough set of tests.

Proposal: Add general purpose host memory allocator reference to cuIO with a demo of pooled-pinned allocation.

1

This PR adds a new interface to cuIO which controls where host memory allocations come from. It adds two core functions: Addresses https://github.com/rapidsai/cudf/issues/14314 ``` void set_current_host_memory_resource(cudf::host_resource_ref mr); cudf::host_resource_ref get_current_host_memory_resource(); ```...

[FEA] Implement a templated parquet decoding kernel suitable for reuse in micro-kernel optimization approach.

3

As part of the drive towards implementing the micro-kernel parquet decoding strategy, we would like to start centralizing the core parquet decoding loop into a generic templated implementation that can...

[BUG] Crash running parquet reader benchmarks.

1

The PARQUET_READER_NVBENCH crashes (segfault) at exit on some machines. It doesn't seem to happen consistently for everyone, but it tends to be reproducible once it starts happening. To reproduce, run...

[BUG] Parquet column selection by name with schemas including list<struct<X, Y>> does not work.

5

If you have a schema that contains a list-of-struct, selecting a subset of the inner columns doesn't work. Example `list` If the schema for this column was ``` A (list)...

Our external interface to the parquet reader allows the user to specify `skip_rows` / `num_rows` parameters when calling it. Internally, we use the same values. But it is a very...