[C++] See if possible to deduplicate dictionaries in IPC streams in some way

Open asfimport opened this issue 6 years ago • 1 comments

As follow-on work to ARROW-3144, there are cases where a dictionary may be shared by multiple fields in a RecordBatch.

The presumption of arrow::ipc::DictionaryMemo is that there is a 1-to-1 mapping between fields and dictionaries, and dictionary id assignment occurs prior to observing the dictionaries (to know whether or not they are used multiple times), so it may not be feasible, or at least not easy.

Reporter: Wes McKinney / @wesm

_{Note: This issue was originally created as ARROW-5340. Please see the migration documentation for further details.}

May 15 '19 21:05 asfimport

This issue hasn't had activity in a long time. If it's still being worked on, please leave a comment. Otherwise, it will be closed on 23rd June.

Labelled Status: Stale-Warning for tracking.

Jun 21 '25 08:06 thisisnic