Lib with reference ONLY - OASIS circular dependency (and crashes klayout), GDS crashes GDSTK (but klayout OK)

[nmz787-intel]

Did some more digging after my comment on issue #49

It looks like a few issues. Basically my goal is to stream-in my GDSTK-authored GDS/OAS files into Cadence Virtuoso (presumably openAccess format internally, though I'm unsure).

OAS would be nice to use, because of compressed file size. But it appears that GDSTK cannot write OAS references properly, unless the dependency is also in the same lib/file. I've confirmed that OAS files produced by Cadence Virtuoso strmout produces valid files with references-only.

import gdstk
import decimal
unit_value = 1e-6 #micrometer is the unit everyone passes data around with
precision = decimal.Decimal(unit_value) / decimal.Decimal(40000)
reference_dependency = 'my_top_level_cell1'

lib = gdstk.Library(unit=unit_value, precision=precision)
cell = lib.new_cell(reference_dependency)
cell.add(gdstk.rectangle((0, -10), (150, 10)))
lib.write_oas('my_reference_dependency.oas')
lib.write_gds('my_reference_dependency.gds')


lib2 = gdstk.Library(unit=unit_value, precision=precision)
cell2 = lib2.new_cell('my_reference_based_cell')
cell2.add(gdstk.Reference(reference_dependency))
lib2.write_oas('my_reference_usage.oas')
lib2.write_gds('my_reference_usage.gds')

for file_type, open_method in [
                               ('gds', gdstk.read_gds), #  comment this line to avoid segfault
                               ('oas', gdstk.read_oas)]: # this line ends up printing: in cell (my_reference_based_cell) we have ref to (my_reference_based_cell)
    filename_to_check = f'my_reference_usage.{file_type}'
    print(f'\n\nOPENING ({filename_to_check}) to check cellnames and reference names')
    o = open_method(filename_to_check)
    for c in o.cells:
        for r in c.references:
            print(f'in cell ({c.name}) we have ref to ({r.cell.name})')
print('done')

Hopefully this table summarizes my findings clearly enough!

	OAS created by GDSTK	GDS created by GDSTK	OAS created by Virtuoso
read by GDSTK	OPENING (my_reference_usage.oas) to check cellnames and reference names in cell (my_reference_based_cell) we have ref to (my_reference_based_cell) done	OPENING (my_reference_usage.gds) to check cellnames and reference names [GDSTK] Missing referenced cell my_top_level_cell1 /mnt/c/Users/nathan/AppData/Roaming/JetBrains/PyCharmCE2020.1/scratches/scratch_23.py:25: RuntimeWarning: Missing reference. o = open_method(filename_to_check) Segmentation fault (core dumped)	>>> gdstk.read_oas('references_only.oasis') [GDSTK] Missing referenced cell ref1_cell [GDSTK] Missing referenced cell ref2_cell [GDSTK] Missing referenced cell ref3_cell [GDSTK] Missing referenced cell ref4_cell :1: RuntimeWarning: Missing reference. Segmentation fault (core dumped)
read by Virtuoso	Unable to load file {my_reference_usage.oas}. (Circular reference: cell 'my_reference_based_cell' referenced itself. reading aborted.)	OK	OK
read by kLayout	internal error: ../../../src/db/db/dbLayout.cc:1330 topological_sort () was not true	OK	OK

[jatoben]

@nmz787-intel Could you share the version of gdstk you're currently using? The segfault when reading a GDS file looks similar to one fixed previously, in https://github.com/heitzmann/gdstk/pull/68.

[nmz787-intel]

0.8.0

[nmz787-intel]

I see each release since then has some fixes for references, upgrading now

[jatoben]

Thanks! If you're able to upgrade to 0.8.1 or later, I believe that will resolve the segfault.

[nmz787-intel]

Segfault is resolved, but discrepancy between reloaded GDS and OAS is still there.

Had to update the print statements in code posted before: from: print(f'in cell ({c.name}) we have ref to ({r.cell.name})') to: print(f'in cell ({c}) we have ref to ({r.cell})')

Output now:

OPENING (my_reference_usage.gds) to check cellnames and reference names
[GDSTK] Missing referenced cell my_top_level_cell1
/mnt/c/Users/nathan/AppData/Roaming/JetBrains/PyCharmCE2020.1/scratches/scratch_23.py:25: RuntimeWarning: Missing reference.
  o = open_method(filename_to_check)
in cell (Cell 'my_reference_based_cell' with 0 polygons, 0 flexpaths, 0 robustpaths, 1 references, and 0 labels) we have ref to (my_top_level_cell1)


OPENING (my_reference_usage.oas) to check cellnames and reference names
in cell (Cell 'my_reference_based_cell' with 0 polygons, 0 flexpaths, 0 robustpaths, 1 references, and 0 labels) we have ref to (Cell 'my_reference_based_cell' with 0 polygons, 0 flexpaths, 0 robustpaths, 1 references, and 0 labels)
done

kLayout issue is still the same when reading OAS file.

[heitzmann]

@nmz787-intel Indeed it looks like there might be a problem with saving OASIS files with missing dependencies. I'll look into that and keep you posted in this issue.

[heitzmann]

I'll consider the issue solved, as there's been no further comments after the commit.