Qt-less canvas discussion

[klayoutmatthias]

This is a continuation of the related discussion from here: https://github.com/KLayout/klayout/issues/1069

[joamatab]

@proppy @tvt173

[klayoutmatthias]

The qtless-canvas branch now is in a fairly stable state. With the Python wheel available in the Artifacts section you should be able to do something like this without Qt:

import klayout.lay as lay
import klayout.db as db

lv = lay.LayoutView()
lv.load_layout("samples/lvs/ringo.gds", True)
lv.load_layer_props("samples/lvs/tech.lyp")
lv.set_config("background-color", "#300000")
lv.max_hier()

# signature same as get_image_with_options, but returns a PixelBuffer
# object which substitutes QImage in the non-Qt case:
pxbuf = lv.get_pixels_with_options(800, 500)
png_string = pxbuf.to_png_data()

with open("pixels.png", "wb") as file:
  file.write(png_string)

Explanation: The "lay" module contains a superset of the layout viewer/edtiro API. Specifically LayoutView is a non-Qt class now and offers much of the functionality of LayoutView. Except image export is done via a new class named PixelBuffer instead of QImage. The PNG read and write abilities come from libpng. It's also possible to turn an image into a byte string in PNG format as shown in the example above.

This however, is just the beginning. LayoutView offers a dynamic API which should basically be able to provide background drawing and mouse event simulation which eventually enables zoom, drawing of rulers and even editing the same way you are used from the desktop application. I'm currently evaluation ways how this can be utilized.

Matthias

[proppy]

neat!

. Except image export is done via a new class named PixelBuffer instead of QImage. The PNG read and write abilities come from libpng. It's also possible to turn an image into a byte string in PNG format as shown in the example above.

Would you be open to us adding support for the IPython/Jupyter display protocol by adding _repr_* method to the layout class as described in https://ipython.readthedocs.io/en/stable/config/integrating.html#rich-display (the nice things is that it doesn't include any dependencies to IPython, it's just relying on mime type and bytestring representation and do the usual :duck: typing thing), happy to send a PR against your branch if you like the idea!

This however, is just the beginning. LayoutView offers a dynamic API which should basically be able to provide background drawing and mouse event simulation which eventually enables zoom, drawing of rulers and even editing the same way you are used from the desktop application. I'm currently evaluation ways how this can be utilized.

Neat, that would allow for easy integration with https://holoviews.org/reference/containers/bokeh/DynamicMap.html as (I think?) we discussed w/ @tvt173 before.

[tvt173]

this is awesome @klayoutmatthias ! thanks for doing this and providing the example to start from!

indeed, it will be even more compelling once we are able to integrate the other mouse events and have some sort of widget we could integrate into a notebook or web page. is there a framework you're thinking of using to help accomplish this? it seems like what you're doing is fairly "raw"... you still have klayout handle the rendering of the layout into an image and responses to any events... maybe something fairly lightweight like this would be appropriate? https://github.com/martinRenou/ipycanvas

once we have some sort of widget established, it would indeed be cool, as @proppy suggests, to have that widget registered to automatically show up in a notebook when it displays a layout

[tvt173]

what are your plans for releasing an initial version of this @klayoutmatthias ?

@joamatab , even in the current state of this, i think it would be a nice alternative to Component.plot() in gdsfactory, i.e. for building documentation. you would then get images of the components that perfectly match how they look in klayout (what designers are most accustomed to seeing)

[klayoutmatthias]

I have just merged the changes into master. It will go into 0.28, but not in 0.27.x. It's a too big change for a minor release.

A rough guess of mine was end of July for 0.28.

[tvt173]

thanks @klayoutmatthias ! i look forward to playing with it

[klayoutmatthias]

Maybe you want to take a look at this: https://github.com/klayoutmatthias/canvas2canvas

It's a demo for a combination of HTML server using the KLayout Python module and an HTML5 client using a Canvas to display the KLayout screen. Mouse events are sent from the client to the server through a WebSockets connection which in the other direction transmits the KLayout screen image.

Right now, zoom and select are working already on a fixed demo layout.

[tvt173]

Hi @klayoutmatthias , thanks for providing the example and instructions! I was just trying to get it running, however I get the following error:

    return await cast(
  File "/home/troy/canvas2canvas/./server.py", line 37, in connection
    self.layout_view.load_layout(self.url)
RuntimeError: Invalid number of arguments (got 1, expected 2/3/4) in LayoutViewBase.load_layout

Looks like maybe the signature of that function changed since the time this example was written?

[tvt173]

nevermind! didn't realize i needed to be on the qtless-canvas2 branch. it's working now! this is awesome @klayoutmatthias ! I'll play around a bit and let you know if I have any feedback

[klayoutmatthias]

Very good :)

I'll merge the branch soon, but as this is a pretty big change I want to give it some testing.

I have meanwhile enhanced the demo app somewhat so it offers some more options and will also push that code soon.

Best regards,

Matthias

[proppy]

@klayoutmatthias that's awesome work.

I just gave a try, I had to apply the following tiny patch for it to run:

diff --git a/server.py b/server.py
index 32dc238..472c03f 100755
--- a/server.py
+++ b/server.py
@@ -54,7 +54,7 @@ class LayoutViewServer(object):
   async def connection(self, websocket, path):
 
     self.layout_view = lay.LayoutView()
-    self.layout_view.load_layout(self.url)
+    self.layout_view.load_layout(self.url, True)
     self.layout_view.max_hier()
     await websocket.send(json.dumps({ "msg": "loaded", "layers": self.layer_dump() }))
 
@@ -143,4 +143,3 @@ class LayoutViewServer(object):
 
 server = LayoutViewServer(layout_url)
 server.run()

I'm curious to get your thought on the best way to integrate this with a dynamic viewport, the sample shared seems to send raw mouse event to the underlying layout view but I think integration with package like holoview / datashader will likely require us to manage rectangle viewport updates instead.

I looked at the LayoutView implementation and it seems that there are function to manage the viewport like goto_window https://github.com/KLayout/klayout/blob/348faa64b3893433c3693c8d68dad9e484a32938/src/laybasic/laybasic/layLayoutViewBase.h#L1536 but those don't seem to be exposed thru the current Python interface

I also found get_pixels_with_options https://github.com/KLayout/klayout/blob/348faa64b3893433c3693c8d68dad9e484a32938/src/laybasic/laybasic/layLayoutViewBase.h#L913 which seems to accept a target_box parameter, but I'm wonder if at that point it wouldn't be easier to expose the underlying Canvas object? https://github.com/KLayout/klayout/blob/348faa64b3893433c3693c8d68dad9e484a32938/src/laybasic/laybasic/layLayoutCanvas.h

Thoughts?

[proppy]

Also do you know if there is a way to get a raw array out of a https://github.com/KLayout/klayout/blob/348faa64b3893433c3693c8d68dad9e484a32938/src/laybasic/laybasic/layPixelBuffer.h w/ the Python bindings? (I'd like to avoid getting the pixel one by one, or converting from a base64 representation)

[klayoutmatthias]

@proppy Are you sure you used the qtless_canvas2 branch? The second argument to "load_layout" is not required in this branch.

You can change the view's coordinates using "zoom_box" in LayoutView. This is equivalent to "goto window". The screen dimensions are set with "resize".

You can get the raw data from PixelBuffer as a PNG binary string with "to_png_data". I can basically also give you the raw binary data, but IMHO that bears a risk of interpretation issues (endianess, byt packing, stride etc.) and after all you want to send small data packages only over the network. PNG is also well compatible with the JavaScript API.

Best regards,

Matthias

[klayoutmatthias]

Sorry, I just came across this post. I though I responded, but it looks like I forgot.

The get_pixels_with_options can deliver a PNG format byte array through PixelBuffer#to_png_data. That is both compact and portable.

I don't want to expose too much of the C++ API. This will restrict the development as backward compatibility mandates a conservative approach. That is why I do not simply bind C++ objects to Python using swig for example. I value API stability and still want to stay flexible in terms of continuous improvement.

I think the raw event interface already is very useful and I will try to use it for other applications than yours. So that is the API I am going to support.

Matthias

[proppy]

@klayoutmatthias I cleaned up the notebook demo we worked on at FSiC and uploaded it here: https://colab.research.google.com/gist/proppy/6956a38ce1a69c710719c7edb40377c6/klayout-playground.ipynb

[klayoutmatthias]

Thanks for this discussion. It's time to close it :)

[proppy]

Now that klayout 0.28 is out, here is a small snippets on how to make it work:

python -m pip install klayout

import klayout.db as db # necessary to get the gds plugin loaded
import klayout.lay as lay

db.Technology.clear_technologies()
tech = db.Technology.create_technology('gf180mcu')
tech.load('gf180mcu.lyt')

lv = lay.LayoutView()
lv.load_layout('gf180mcu_fd_sc_mcu9t5v0__addf_1.gds')
lv.max_hier()
pixels = lv.get_pixels_with_options(500, 500)
with open('gf180mcu_fd_sc_mcu9t5v0__addf_1.png', 'wb') as f:
    f.write(pixels.to_png_data())

https://colab.research.google.com/gist/proppy/11ef0a0a59ac291dd99f5fc6a9f3d8dc/klayout-0-28-load_layout-1146.ipynb

[tvt173]

thanks @proppy ! would be awesome to revisit and flesh out @klayoutmatthias 's interactive demo as well. images will be great for things like auto-generated documentation, but an interactive visualizer would be much more valuable i think for most applications

[proppy]

@tvt173 yep, I'm in the process of updating https://github.com/chipsalliance/silicon-notebooks to rely on klayout instead of gdstk (see https://github.com/chipsalliance/silicon-notebooks/issues/30#issuecomment-1345721364).

Maybe we could continue the discussion there on how to make it interactive?

[proppy]

@klayoutmatthias is there a way to hide the cell frame using the python API, just like you can do in the 'View / Show Cell Frames' menu?

[klayoutmatthias]

Yes, all these options are accessible through LayoutView#set_config. In your case this is:

pya.LayoutView.current().set_config("inst-visible", "false")

Tip: look up configuration keys and values in ~/.klayout/klayoutrc.

Matthias