ion-python
ion-python copied to clipboard
Add simpleion support for roundtripping streams with unavailable symbol table imports
When a shared symbol table is not found, a stream can still be read; the symbols referenced from the unavailable imports simply have "unknown" text (e.g. "$128"). When this happens, it should be possible to re-write the stream, referencing the not-found imports, so that in the future (or down the pipeline) other readers can process the full-fidelity data if they have access to that import.
This is not currently possible with simpleion. All keys (field names) are strings, so field name tokens with unknown text will all collapse to None
. To support this feature, all symbol tokens with unknown text need to be preserved as SymbolToken
s (with None
text, a non-None
SID, and a valid import location). A tailored implementation of dict
is likely necessary for full support. See also: #36 .
Successful resolution of this issue will involve removal of the following files from the test_vectors.py
skip list:
good/item1.10n
good/nonequivs/symbolTablesUnknownText.ion