Support auto-naming label's based on menu choices

[pmrowla]

trafficstars

Related to #19

this will require the menu parsing logic being moved into the API, but once that is done we can start re-naming labels https://github.com/pmrowla/pylivemaker/blob/9a25cfb8a4ba4bfb86512f7d7f3fcb4a145b74ca/livemaker/lsb/command.py#L445-L454

When LM compiles an lsb, they strip the original label names and replace the name field with a hex string for the command number. But we can change the label name to whatever we want, to make naming better in things like lmlsb extract, lmlsb extractcsv, etc.

A common use case is for VNs to have route selection or chapter selection options. In livemaker games, this is usually handled by having a a "chapter select" or "route select" lsb, that will contain menus that jump to each chapter. It's usually easy to tell whether or not a game does this by looking at the call graph generated by lmgraph.

So basically if you have a game where 0A.lsb contains a big tree of menus that point to LSB files, with menu choices that equate to

• "A's route" -> Jump(01.lsb)
• "A's ending" -> Jump(10.lsb)
• "A bad end" -> Jump (11.lsb)
• "B's route" -> Jump(02.lsb)
• ...

We can use those menu choices to rename scenarios in each lsb.

So after running something like lm-autolabel --menu=0A.lsb 01.lsb 02.lsb ... The export scripts would properly named scripts/lines A's route-1.lns, A's route-2.lns, B's route-1.lns and so on (both for lns scripts or csv rows)

we could also take advantage of the call graph logic and actually label scripts within lsb's based on the order they are run instead of using the order they appear in the lsb (which are usually not the same).

and renaming labels should not break anything scripting-wise, since in the LM compiled lsb, lookups/references are always done by command ID #, and not by label name

[pmrowla]

this would also work for any arbitrary menu, so you could use things like cg-mode selection menus, in-route choices, etc as long as the menu has associated text to use as the new label name

[Stefan311]

Instead of taking the labels directly from the menu texts, I would prefer to implement an option to change labels manually. The labels from menu texts could be misleading or even appear several times with different labels.

It would be helpful to be able to rename labels for debugging and reverse engineering. These labels could be displayed as comments on jump commands or text blocks. In order to achieve this, you need a list of all labels for the entire project.

1. Search all LSB files for labels, write the labels to a file.
2. The labels whose function you already know are edited in the list file.
3. Each lmlsb function that could resolve labels uses this file as an additional parameter.

For this you need central API functions to store, load and search for labels. I saw that there is a "LabelReference" class, and in which "lookup" functions are already available. So far I have not been able to find out the exact function of this class. Is it already intended for resolving labels?

If you don't mind, I would try to implement a extractlabel/insertlabel function and a API for central label storage and resolving.

[pmrowla]

you can already manually rename labels in the API. https://github.com/pmrowla/pylivemaker/blob/de491e94bc9e5c47f3884cb739c0fed05ab79bf4/livemaker/lsb/command.py#L445

when iterating through the commands in an LSB, if you edit the Name field for a Label command and save the LSB, it will rename the label.

the end result is that if you originally something like like

   9: Label 000000F8
  10: Calc __メッセージ終了 = 0
  11: TextIns <livemaker.lsb.novel.TpWord object at 0x10b4796d0> "メッセージボックス" 1 0 0

and then do something like

label = lsb.commands[9]
label["Name"] = "chapter 1 start"
...<save lsb>

the next time you dump the LSB it should show

   9: Label chapter 1 start
  10: Calc __メッセージ終了 = 0
  11: TextIns <livemaker.lsb.novel.TpWord object at 0x10b4796d0> "メッセージボックス" 1 0 0

label lookup/resolution by name hasn't been fully implemented yet, because up until now there was no real need for it (since normally LSBs only need to do label lookup by command index, and not by label name).

basically, these functions need to be able to open external lsb files, find the appropriate Label command, and get the name label

https://github.com/pmrowla/pylivemaker/blob/de491e94bc9e5c47f3884cb739c0fed05ab79bf4/livemaker/lsb/command.py#L160-L182

and yes, ideally it would be implemented so that there's a central cache of known labels, so we dont have to keep repeatedly opening new lsb files

[pmrowla]

as far as the lookup api goes, I actually already started doing it as a part of moving the menu stuff into the core api (since the destination for a menu choice is relevant), but I wasn't planning on writing any cli tool to do mass renaming of labels.

So I guess maybe just wait until I merge the api changes and then go from there on your label renaming tool?

[pmrowla]

and yeah, auto-naming based on menus is really only relevant for specific use cases, mainly:

A common use case is for VNs to have route selection or chapter selection options. In livemaker games, this is usually handled by having a a "chapter select" or "route select" lsb, that will contain menus that jump to each chapter. It's usually easy to tell whether or not a game does this by looking at the call graph generated by lmgraph.

basically it will only work well for w/route or chapter select menus, or from the CG mode menu.

[Stefan311]

You know IDA, right? It names subroutines primary by it's address. For better understanding the code, you can rename every sub, and every call to this sub is automatic renamed too. That's what I would have for the pylm tool too. It is no need in patching the labels in the actual LSB file. It's only need for dump command.

[pmrowla]

I guess I'm confused as to what you are asking for, since capital-L Label is an actual LSB command type, and Labels are what livemaker uses as branch (Jump/Call) targets. In actual (original) project LSC files, labels have useful string names, but they get compiled out into hex strings when livemaker generates a binary LSB.

Do you just want to be able to assign a (lowercase-L) label to arbitrary commands in an LSB (like comments), and see that in a dump?

[Stefan311]

I guess I'm confused as to what you are asking for, since capital-L Label is an actual LSB command type.

Do you just want to be able to assign a (lowercase-L) label to arbitrary commands in an LSB (like comments), and see that in a dump?

The main goal is to rename actual Label (yes uppercase!) back to a useful LSC name. But since you can every name -also not unique- names, you cannot replace the Jump/Call target. They have to remain command id. I would show the Label on Jump/Call as comment only.

[pmrowla]

ok, so yeah all you really need to do is just figure out how you want to rename Labels and put that into a cli tool.

I can add a --resolve-labels or something option for lmlsb dump that resolves labels into string names. so for something like

0001.lsb
---
1: Jump 0002.lsb:5

0002.lsb
---
5. Label foo

dump for the first file would now show

Jump 0002.lsb:5   # foo

[pmrowla]

since the CSV format (#42) now uses label name as a column, we could probably also just add a flag for insertcsv that does "replace label for each scenario with the current contents of label column from CSV", and you can just rename scenarios as needed in your CSV.

[Stefan311]

Yes, that's it. You can use the renamed labels on many places.

[Stefan311]

I do think it's NOT useful to finally patch the Label into the final LSB file. We need the label only for reverse-engineering, debugging or translation. In the final LSB file the Labels could also be removed.

[pmrowla]

Is there a particular reason you are against patching them into the lsb? Setting the strings in the lsb doesn’t affect anything other than adding a few bytes that get compressed later anyways.

The new comment-like things that would go in the dump would not actually go into an lsb, we would just be replacing the existing Name field in Label commands. I don't see how that is a bad thing, given that Label commands do already have strings in the Name field (but they are just meaningless hex strings).

iirc, when you make an actual livemaker project, intermediate LSBs (that are used for debug builds and live testing when editing a project) also keep the original useful strings in the LSB. They are only stripped for final release builds. I'll have to double check in ghidra but I vaguely remember seeing the functions that do label resolution by name in the release-build interpreter exe. So if we really wanted to, we could even use named references instead of command index references in jumps/calls in patched LSBs.

This isn't a particularly useful thing to have right now, but in theory we could build our own LSBs to add in things like our own route/scene selection menus, which I could see being a useful thing to have for debugging a translatation patch.

we could still strip strings on release, but for our purposes we really don't even have any reason to have separated debug/release builds. for livemaker itself it makes sense when they are trying to prevent people from ripping scripts and assets, but in our case we have already bypassed that and have no real reason to strip anything

[pmrowla]

fwiw there's also nothing preventing us from inserting comments into an lsb either, there is an actual Comment command class, and there's a Mute field in every command, that is used to mark any command as disabled/comment out for debugging purposes.

[pmrowla]

iirc, when you make an actual livemaker project, intermediate LSBs (that are used for debug builds and live testing when editing a project) also keep the original useful strings in the LSB. They are only stripped for final release builds.

Yeah so you can actually this being used in the system LSBs. If a jump/call target is referenced by filename.lsb:..., it treats the subsequent ref as a command index. If a jump/call target is referenced by filename.lsc:... it treats the reference as a label name, and the engine does the lookup. The actual .lsb/.lsc extension is ignored when referencing pages, and the LM engine will use whichever lsb or lsc file is available in the game (or LM project) directory. And compiled system LSBs still use the .lsc:... name based label lookups.

Ex from ノベルシステム\プレビューメニュー\■選択実行.lsb:

...
  36:     PrevMenuNew "選択メニュー" @ParamStr[0] @ParamStr[1]   10000   0 "ノベルシステム\プレビューメニュー\■選択実行.lsc:決定" 1 0 "ノベルシステム\プレビューメニュー\■選択実行.lsc:クリック" @ParamStr[6] im  "ノベルシステム\プレビューメニュー\■選択実行.lsc:ホバー"       0    1   
...
 114: Label クリック
...

the the args for PrevMenuNew include labels for commands (click and hover) that will be run during menu execution on click/hover, and the commands are referenced by label name rather than by line number.

I don't see any real reason for us to avoid taking advantage of this feature

[Stefan311]

I am not against patching the labels at all. It's just useless for the player.

Reasons against label lookup instead of command id? Maybe for performance reasons? Well, it should not make high differences. Using label lookup would have one advantage: we can insert or remove commands without need to recalculate Jump/Call targets.

[pmrowla]

I don't think it will affect performance at all, given that all of LM's system LSB's are already doing name lookups all the time. And yes, obviously it's not useful for a player, but it would be useful for a library/API made for messing around with LSB files (which is what pylm is).