standardize CSV format (columns)

[pmrowla]

trafficstars

For 1.0, CSV column formats between menu CLI tools and scenario text CLI tools should be consistent. For menu CSV specifically, this will break backwards compatibility with <1.0 versions.

TODO

• [x] scenarios
• [x] menus
• [ ] unified extract/insert CLI commands

Related to #37 .

Most likely format will be:

Pylm Identifier	Label Name	Context	Original Text	Translated Text

• Pylm Identifier - ID for internal pylm use
  • Formatted like pylm:<type>:<filename>:<cmd_id>(:...)
  • For menus this will look something like menu:<lsb_id>-<cmd_id>:<jump_id> (this one subject to change, I think we can actually name menus based on labels similarly to how scenario scripts are named)
  • For scenarios this will look something like pylm:text:00000006.lsb:8:0
• Label Name - String label name (if there is a label associated with an object)
  • The end result of this is that when extracting a full LSB in execution order, you will get an properly ordered CSV that includes scenario names/labels at the start of each possible jump (branch) target
• Context - Optional context information that may be useful for translators
  • For menus this may be something like a named menu label or the last couple lines of text before the menu
  • For scenarios this will be speaker name for games which use the speaker name system event
• Original text - Original (JP) text
• Translated text - Translated text

[pmrowla]

Basically the idea is to have a format that allows for potential future translatable object types (beyond menus and scenarios), and that will still handle the case where user appends any type of menu/scenario/etc CSV generated in pylm to a single CSV file.

Having a single ID column gets rid of the issue where different object types may need a different # of parts to identify it.

[pmrowla]

This will also allow us to potentially have a single extract... and single insert... command that recursively searches a project dir for any (non-system) LSB, and automatically extract/insert all translatable menus/text/etc at once, rather than needing separate CLI tools for each translatable object type.

[Stefan311]

You could also make the Filename part of the Pylm Identifier. I also like the idea of having one extract or insert command for everything, but the single functions should be still accesable, or single disable-able (like a --disable-menue option?)

[pmrowla]

You could also make the Filename part of the Pylm Identifier.

Yeah, it actually already is part of the identifiers (the part before the first dash is the lsb name w/stripped file extension). But I was thinking it may still be better to have the filename as a column, since that translators using the csv may not actually understand what the identifiers are.

I also like the idea of having one extract or insert command for everything, but the single functions should be still accesable, or single disable-able (like a --disable-menue option?)

Yeah this will be in it. Unified command would have flags for specifying what types of content. Something like

--scenario/--no-scenario
--menu/--no-menu
<more new translatable types...>

with sane defaults, and a --all convenience option

[pmrowla]

minor issue I noticed while testing #48

Since we use QUOTE_MINIMAL when exporting csv, if a text field starts with a non-alphabet character some spreadsheet programs (I tested in excel and google sheets) try to parse it as a formula expression, and then display a formula parsing error in the cell.

In particular, - is an issue: a common use case will be text blocks starting with -<honorific>, where the text block would be preceded by script event for inserting a variable (like player name).

we should probably be quoting things so that they are properly treated as text strings in common spreadsheet apps

see: https://superuser.com/questions/318420/formatting-a-comma-delimited-csv-to-force-excel-to-interpret-value-as-a-string