gdstk icon indicating copy to clipboard operation
gdstk copied to clipboard

Read Large OAS file

Open XDean opened this issue 2 years ago • 2 comments

I have .oas file that larger than 5GB with billions polygons.

When I use gdstk to read, it used minutes time and memory > 5GB.

Is there a way to handle the large oas without read it all into memory at first time? Something like numpy.memmap.

Or is there a way to read it as several parts?

XDean avatar Mar 10 '22 03:03 XDean

@XDean at this point, there is no way to read it partially. In the future, I plan to add a filter functionality similar to read_gds, but there are other issues I'm working on before that.

heitzmann avatar Mar 10 '22 11:03 heitzmann

I think there is always memory problem while reading large oas files, it consumes very big memory resources suddenly.

MarkMa1990 avatar Jul 09 '22 20:07 MarkMa1990