klayout
klayout copied to clipboard
KLayout
Can't run Py Macro: opens DRC DB browser instead
I can't run Python or Ruby macros from the Macro Development window - clicking "Run this script" erroneously opens the "Marker Database Browser" (which usually pops up after running a DRC script - which I haven't actually done on this computer).
The Python script never runs.
I can run python scripts from the CLI, eg. from Terminal/BASH: ./klayout.app/Contents/MacOS/klayout -r /Users/demis/.klayout/pymacros/DieLabelling_v2.lym does run. That .lym file was created in the GUI Macro editor - I just can't run it from the GUI.
I also upgraded from 0.26.5 --> 0.26.8 and still get the same behaviour.
My scripts are initialized and saved into my home folder as "local" scripts.
I'm on MacOS 10.14, so they're stored in ~/.klayout/pymacros.
Any suggestions? Thanks, @Kazzz-S - Mac-specific issue perhaps?
Hi
May I have your LYM or a similar one that can reproduce your problem? I want to test it on different platforms including Macs, Linux, and Windows.
Also, please confirm that you are using ST-klayout-0.26.8-macOS-Mojave-1-qt5MP-RsysPsys.dmg. Or, have you built KLayout from the source code?
Thanks, Kazzz-S
Indeed, I am using the most recent KLayout install, my downloaded DMG has the same filename you mention: ST-klayout-0.26.8-macOS-Mojave-1-qt5MP-RsysPsys.dmg I don't think it's the LYM file itself - any new template project I generate fails to launch the python script, but instead shows the post-DRC-run "Marker browser". But I will attach the scripts as ZIP below.
Also I notice that an LYM with "add to menu" does not show up in the menu anywhere that I can figure.
I deleted my prefs and relaunched KLayout: I just renamed the ~/.klayout folder, along with ~/Library/Preferences/klayout.de.plist, and with the Macro Development menu, I made a dummy Python script. Now this script actually does run with "run current script", and the code I inserted print("I'm working") successfully echos the text to the Python Console.
Then I tried putting my original LYM file back into the newly created ~/.klayout/pymacros folder and running it, but Even with DieLabelling_v2.lym shown as frontmost script in "Macro Development", the "Run Current Script" button still runs only new_python_macro.lym, and echoes the string from that macro. The echo'd string even updates if I change the print() text in new_python_macro.lym.
Why does it not run the "DieLabelling_v2.lym" file, even when it's in "front" when I hit the "run current script" button?
Screenshot, while updating/saving the print() text of new_python_macro.lym each time, showing console text indicating it is Not running the desired frontmost script:
On the plus side, the DRC "marker browser" does Not launch - but perhaps this helps to elucidate the problem. The run script button is still running something other than the frontmost script. So on my previous preferences, perhaps it was running an old DRC script that was, somehow, queued up to run in the background. Just an idea.
Hi,
Thank you for sharing your LYM. Digesting your LYM (DieLabelling_v2.lym), I have created four variations, out of which three (A, B, C) worked, as shown in the screenshots below.
1. macOS Mojave (your version)
2. macOS Catalina (ST-*.dmg)
3. macOS Catalina (MacPorts.dmg)
4. Linux Mint 19
5. Mac-OSX Yosemite + KLayout 0.25.9
Images are not captured, but the three (A, B, C) macros also worked like other macOS platforms.
That's it for today. Tomorrow or the next day, I will upload the modified PYA codes and Windows 10 screenshots.
Kazzz-S
Hi,
- I have attached the modified macros.
- Referring to the table below, please place the files in the specified directories.
- You may need to edit the GDS file locations further.
- When you start KLayout, you will see the three menu entries shown in the above screenshots.
- Their actions are identical.
| Srl.No. | File Name | Install Location |
|---|---|---|
| 1 | DieLabelingA_KazzzS.lym | $HOME/.klayout/pymacros/ |
| 2 | DieLabelingAX_KazzzS.lym | --ditto-- |
| 3 | DieLabelingB_KazzzS.py | --ditto-- |
| 4 | Nanofab.lym | $HOME/.klayout/python/ |
| 5 | Nanofab_DieLabelingC.py | --ditto-- |
- Srl.No.2 (DieLabelingAX_KazzzS.lym) does not work. It looks like a bug (?) of the XML parser.
- The
<text> ~~~ </text>body contains a raw "less than <" character in a "if statement," which looks interpreted as a beginning of a new XML tag. - That part is changed in Srl.No.1 (DieLabelingA_KazzzS.lym) to fool the parser.
Your original problem
- While testing the modified macros with
2. macOS Catalina (ST-*.dmg), I experienced a similar problem as you. But a different dialog showed up unexpectedly. I missed capturing the images, sorry :-(. - If I remember correctly, it was a dialog for setting the shape properties.
- After cleaning the macros, it never happens even in an old environment
5. Mac-OSX Yosemite + KLayout 0.25.9. - I think 50% of the causes were somewhere in our codes. The mechanisms I estimate are as below.
Related Issue-1
Related Issue-2
- An LYM script has some problems, which trigger memory corruption in the Python interpreter.
- In the GUI mode, everything is controlled by a giant event loop.
- Therefore, small memory corruptions may lead to unexpected behavior of GUI.
- macOS looks very sensitive to this issue.
- The damaged memory locations are different in your case and my case.
- So we have seen various dialogs popped up suddenly.
I hope my comments helped you. Regards, Kazzz-S
Thank you for looking into this Kazzz-S, I will try the modified LYM files this week. However I'm not understanding the root-cause in my case - none of my scripts have a "less than <" symbol in the python script, isn't that right?
Even creating a brand new pymacro LYM file from the built-in templates created the same behavior.
So what do you think could be the potential "LYM problems" in the script file that cause a corruption in the python memory space (with no fault detection)?
Hi,
Indeed, it is challenging to find the root cause of this kind of problem. In my gut feeling, Related Issue-1, which looks macOS specific, is suspicious. When I attempt to insert menu items to my desired locations, they appear in different places. This fact implies improper memory access is happening. The bad news regarding this is that the Valgrind tool supports up to macOS 10.13 only so far. I have macOS 10.13 (High Sierra), but it's a virtual machine and too sluggish even in regular operation. I'm sure that the Valgrind would not run in the environment.
- In your initial LYM,
<menu-path>DieLabelling</menu-path>does not show up anywhere on the menu. - In my modified LYMs, by clearing
<menu-path> ~~ </menu-path>,which means 'use the default "Macro" menu,' the menus show up, and LYMs became stable.
From these experiments, I guess that failing to insert a menu to an arbitrary location (other than "root") triggers a sequence of problems. An unintended pop up of a dialog is a side effect of those.
Kazzz-S
Thanks for explaining how you fixed it. Testing what you pointed out here, via the GUI, confirms what you found regarding menu insertion:
When I leave the text-field Macro Development > Macro Properties > **Menu Path** blank, the script shows up int he Menu, and runs with the "Run Now" button.
However the "Group name" field doesn't appear to have an obvious effect, instead the full script path is always shown in the menu:
What is the expected behavior of the "Group name"? Is it meant to be a sub-menu of the Macros menu? It appears to have no effect as far as I can tell.
––––––––––––––––––––––
As you found, if I fill in the "Menu Path" field in the GUI, then
- the script doesn't show up in the menu anywhere.
- The script IS able to be run (only tested via
print('hello world") commands).
The behavior is reversible - If I remove the text from Menu Path field, the script does show up in the menu (as a path/to/macro.lym).
––––––––––––––––––––––––––––––––––
Now that I have removed my ~.klayout folder and ~/Library/Preferences/Klayout* files, I never see the DRC popup anymore. So perhaps my prefs were corrupted?
I have attached a ZIP file of backups of those preference locations (prior to renaming them), in case locating the preference file corruption could help narrow down the memory fault.
klayout prefs 2020-10-14.zip
dot_klayout_2020-10-14.zip
–––––––––––––––––––––––––––––––
On the other issue of why I couldn't run the frontmost Tab, I just discovered the "Run script from the current tab" button, which now allows me to change which script Tab gets run - so half of my issue was "user error", as we call it here in the NanoFab!
Hi,
Good to see you could proceed!
I've gone through the binary property list file (klayout.de.plist, in your case) by converting it to an XML file by:
plutil -convert xml1 klayout.de.plist
The contents are 1) the last used directory, 2) Window sizes, etc. only. So nothing looks special. Refer to this for more details.
Regarding the Group name, I've never used it. As per the KLayout document, however, yes, it is for grouping some menu items. Referring to https://www.klayout.de/doc-qt5/about/macro_in_menu.html, I've tried as follows.
- "DieLabelingA_KazzzS.lym" === copy ===> "DieLabelingA1_KazzzS.lym"
- "DieLabelingB_KazzzS.py" === copy ===> "DieLabelingB1_KazzzS.py"
Then edited the header parts as below. With this approach, it looks explicit group-name is not necessary. GroupName.zip
DieLabelingA1_KazzzS.lym
<?xml version="1.0" encoding="utf-8"?>
<klayout-macro>
<description>DieLabeling_A1</description>>
<version/>
<category>pymacros</category>
<prolog/>
<epilog/>
<doc/>
<autorun>false</autorun>
<autorun-early>false</autorun-early>
<shortcut/>
<show-in-menu>true</show-in-menu>
<group-name/>
<menu-path>tools_menu.my_nanofab_group>end("Nanofab").end</menu-path>
<interpreter>python</interpreter>
<dsl-interpreter-name/>
<text>
DieLabelingB1_KazzzS.py
# $description: DieLabeling_B1
# $show-in-menu
# $menu-path: tools_menu.my_nanofab_group.end
Kazzz-S
Thanks for all these analysis you've done.
I think I can clarify at least the "Group name" feature: If you specify a name there (some string which acts as an identifier for the group), you can create horizontal separators for forming menu item groups. As the documentation explains, you should use ".end" (e.g. "tools_menu.end") for the menu path and use the same group name for the menu items you want to put behind the same separator.
Regarding the initial problem: I understand that the incorrect menu path might have been the reason for the unexpected behaviour. So my question is: if you use a proper menu path, such as "tools_menu.end" - does the initial problem still happen? With an incorrect menu path, the script simply does not show up anywhere on my Linux box. But I can't rule out this would mess up anything internal and the weird effect only happens on MacOS.
And I still wonder how the "<" character gets into the .lym files - maybe they were edited manually?
Thanks and best regards,
Matthias
I did not edit the files manually - I was previously only working with the GUI, only Kazzz's comments suggested to open the *.lym file manually.
Which does beg the question - should the GUI parse the string properly to make it work as expected. For example, adding or warning about the ".end" terminator?
On Mon, Oct 26, 2020 at 3:29 PM Matthias Köfferlein < [email protected]> wrote:
Thanks for all these analysis you've done.
I think I can clarify at least the "Group name" feature: If you specify a name there (some string which acts as an identifier for the group), you can create horizontal separators for forming menu item groups. As the documentation explains, you should use ".end" (e.g. "tools_menu.end") for the menu path and use the same group name for the menu items you want to put behind the same separator.
Regarding the initial problem: I understand that the incorrect menu path might have been the reason for the unexpected behaviour. So my question is: if you use a proper menu path, such as "tools_menu.end" - does the initial problem still happen? With an incorrect menu path, the script simply does not show up anywhere on my Linux box. But I can't rule out this would mess up anything internal and the weird effect only happens on MacOS.
And I still wonder how the "<" character gets into the .lym files - maybe they were edited manually?
Thanks and best regards,
Matthias
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/KLayout/klayout/issues/657#issuecomment-716858792, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABI7CRIW3W74LCY4FSDM7HTSMXZ45ANCNFSM4SQWHWCQ .
And I still wonder how the "<" character gets into the .lym files - maybe they were edited manually?
Yes, I used an external text editor and inserted my Python code to the <text>~~~</text> block manually.
Kazzz-S