ion-python icon indicating copy to clipboard operation
ion-python copied to clipboard

Add simpleion support for roundtripping streams with unavailable symbol table imports

Open tgregg opened this issue 8 years ago • 0 comments

When a shared symbol table is not found, a stream can still be read; the symbols referenced from the unavailable imports simply have "unknown" text (e.g. "$128"). When this happens, it should be possible to re-write the stream, referencing the not-found imports, so that in the future (or down the pipeline) other readers can process the full-fidelity data if they have access to that import.

This is not currently possible with simpleion. All keys (field names) are strings, so field name tokens with unknown text will all collapse to None. To support this feature, all symbol tokens with unknown text need to be preserved as SymbolTokens (with None text, a non-None SID, and a valid import location). A tailored implementation of dict is likely necessary for full support. See also: #36 .

Successful resolution of this issue will involve removal of the following files from the test_vectors.py skip list: good/item1.10n good/nonequivs/symbolTablesUnknownText.ion

tgregg avatar Jan 28 '17 01:01 tgregg