Disabling boxes/tables in help output

[jscholes]

I was recently made aware of Cyclopts in this Mastodon post, and really like the look of it! However, I'm running into a bit of a blocker for my use case.

I myself am a blind screen reader user, and it's important that the apps I develop are as inclusive/accessible as possible. For people who can see, this means that the default help output with boxes, rounded corners, tables and the like can be legitimately helpful.

But with screen reading software, the characters don't really translate at all into understandable speech or braille output, and it makes the usage, help and error information generated by Cyclopts quite challenging to read.

What I'd really like is to be able to configure Cyclopts to use completely plain text output, or better yet my own custom help formatter, perhaps in response to a particular environment variable, CLI parameter or config file flag being set. For example with this program:

import cyclopts


app = cyclopts.App()


@app.command
def init():
	"""Initialise a thing."""

	print('Initialised!')


@app.command
def export():
	"""Export a thing."""

	print('Exporting...')


if __name__ == '__main__':
	app()

The default help output looks something like this:

Usage: main.py COMMAND

┌─ Commands ──────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ export     Export a thing.                                                                                          │
│ init       Initialise a thing.                                                                                      │
│ --help -h  Display this message and exit.                                                                           │
│ --version  Display application version.                                                                             │
└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

I'd love to make it look something like this under the appropriate circumstances:

Usage: main.py COMMAND

Commands:
export: Export a thing.
init: Initialise a thing.
--help or -h: Display this message and exit.
--version: Display application version.

This probably looks quite ugly/basic, but it is pretty screen-reader-friendly. This is also a very simple program and there'd obviously be a lot of additional formatting to handle in real world scenarios.

Let me know if I can provide more information. Thanks for sharing your work with us all!

[BrianPugh]

Hi @jscholes!

Unfortunately, currently there isn't really a way to customize the layout of the help-page. This is something that we can add to the future; a first step would be to abstract out the data-gathering from the layout-drawing, which would also probably help out #180.

However, the highest priority feature for me at the moment is autocomplete, and i haven't had too much free time to work on it 😿 . So I'd say this is a "months away" feature rather than "days/weeks away" feature.

[jscholes]

@BrianPugh As a fellow busy human, I completely understand. Appreciate the transparent response, and I wish I had time to help out with it myself.

[ESPR3SS0]

@BrianPugh Hello, I've been tinkering with abstracting / Dividing the Formatting of the Help Panel and the Information Gathering / Rendering. Most tests are passing, but I'm at the point where it might be good to get some feedback on design before I drill down...

Check it out here: https://github.com/ESPR3SS0/cyclopts/tree/abstract_help. If you want to check that out and see how you feel about the flow I'd love to keep working on it.

Given how many changes there are, if this type of design is something that could be included in cyclopts, I'm also wondering how'd you like me to best work in such a way that adding it won't be a pain! Don't want to be a burden with a million edits....

TLDR of the idea...

Spec objects control how the Panel Looks. Entries hold arbitrary information, and control what info a table can show.

The flow then looks like:

1. Define the column Specs with ColumnSpec, table specs with TableSpec and panel spec with PanelSpec.
2. Select what type of columns are included in the table.
3. Build the table with table_spec.build(). This adds the entry information and uses the ColumnSpec to render the info.
4. Build the panel and yield.

ColumnSpecs are responsible for rendering entry information. The entry matched to the corresponding column spec, which optionally applies two sets of functions:

1. Formatter: Used to format the data. I.E) A wrap formatter to wrap parameter names
2. Converter: Used to change the message itself. I.E) Split "--opt1--no-opt1" into "--opt1 --no-opt1", or combine names entry.name + " " + entry.short.

Benefits:

1. Division of the formatting of the HelpPanel and the rendering of the HelpPanel. Mentioned in #50
2. It moved the rending logic out of the HelpPanel object, so new functions can be plugged in and the column ordering / formatting can be adjusted.
3. Are related to and could help #50 and #180
4. Gives the ability to render custom information in addition to 'name', 'short', 'description' and 'asterisk'.
5. The cells are rendered lazily & can use a callable at render time to do "fun things" (i.e. read current disk space and print this in the help panel / read status of some process and render).

Here's a snippet from my branch to give you a snippet of the flow:

# For Parameters:
AsteriskColumn = ColumnSpec(key="asterisk", header="", justify="left", width=1, style="red bold", converters=asterisk_converter)
NameColumn = ColumnSpec(key="name", header="", justify="left",  style="cyan", formatter=wrap_formatter, converters=[stretch_name_converter, combine_long_short_converter])
DescriptionColumn = ColumnSpec(key="description", header="", justify="left", overflow="fold")
# For Commands:
CommandColumn = ColumnSpec(key="name", header="", justify="left", style="cyan", formatter=wrap_formatter, converters=[stretch_name_converter, combine_long_short_converter] )


@define
class AbstractRichHelpPanel:
    """Adjust the Format for the help panel!."""

    import textwrap
    from rich.box import Box, ROUNDED
    from rich.console import Group as RichGroup
    from rich.console import NewLine
    from rich.panel import Panel
    from rich.table import Table, Column
    from rich.text import Text

    format: Literal["command", "parameter"]
    title: RenderableType
    description: RenderableType = field(factory=_text_factory)
    entries: list[AbstractTableEntry] = field(factory=list)

    column_specs: list[ColumnSpec] = field(
        default=Factory(
            lambda self: [CommandColumn, DescriptionColumn]
            if self.format == "command"
            else [NameColumn, DescriptionColumn]
        ,takes_self=True)
    )
    table_spec: TableSpec = field(
        default=Factory(lambda self: TableSpec(columns=self.column_specs), takes_self=True))
    panel_spec: PanelSpec = field(
        default=Factory(lambda self: PanelSpec(title=self.title), takes_self=True))

    def remove_duplicates(self):
        seen, out = set(), []
        for item in self.entries:
            hashable = (item.data.get("name"),
                        item.data.get("short"))
            if hashable not in seen:
                seen.add(hashable)
                out.append(item)
        self.entries = out

    def sort(self):
        """Sort entries in-place."""
        if not self.entries:
            return
        if self.format == "command":
            sorted_sort_helper = SortHelper.sort(
                [SortHelper(entry.sort_key, (entry.name.startswith("-"), entry.name), entry) for entry in self.entries]
            )
            self.entries = [x.value for x in sorted_sort_helper]
        else:
            raise NotImplementedError

    def __rich_console__(self, console: "Console", options: "ConsoleOptions") -> "RenderResult":
        if not self.entries:
            return _silent
        from rich.console import Group as RichGroup
        from rich.console import NewLine
        from rich.text import Text
        commands_width = ceil(console.width * 0.35)
        panel_description = self.description
        if isinstance(panel_description, Text):
            panel_description.end = ""

            if panel_description.plain:
                panel_description = RichGroup(panel_description, NewLine(2))

        # 1. Adjust the format (need to keep default behavior...)
        if self.format == "command":
            commands_width = ceil(console.width * 0.35)
            for i in range(len(self.column_specs)):
                spec = self.column_specs[i]
                if spec.key == "name":
                    spec = spec.with_(max_width=commands_width)
                self.column_specs[i] = spec

        elif self.format == "parameter":
            # Add AsteriskColumn if need be 
            if any([x.get('required', False) for x in self.entries]):
                col_specs = [AsteriskColumn]
                col_specs.extend(self.column_specs)
                self.column_specs = col_specs

            name_width = ceil(console.width * 0.35)
            short_width = ceil(console.width * 0.1)

            for i in range(len(self.column_specs)):
                spec = self.column_specs[i]
                if spec.key == "name":
                    spec = spec.with_(max_width=name_width)
                elif spec.key == "short":
                    spec = spec.with_(max_width=short_width)
                self.column_specs[i] = spec

        # 2.Build table and Add Etnries
        self.table_spec = self.table_spec.with_(columns = self.column_specs)
        table = self.table_spec.build()
        self.table_spec.add_entries(table, self.entries)

        # 3. Final make the panel
        panel = self.panel_spec.build(RichGroup(panel_description, table))

        yield panel

Let me know what you think!

[BrianPugh]

Hey @ESPR3SS0! Thanks for all the work! I'll try and review this tonight!

[BrianPugh]

I only spent about 15 minutes read over it, but overall I like it! Please continue the effort! Some misc minor feedback:

1. Are the Column Names going to be fixed (as they are now)? If so, maybe AbstractTableEntry.data should be a dataclass instead of a dict. I'm also fine with it being a dict if there's good reason.
2. Now that things are getting complicated, we'd probably want to organize everything into a autodecorate/help/ folder/module.
3. Are all the ColumnSpecand TableSpec attributes as similar to rich as possible? It would be good to have consistency there. The build methods suggest they are, just thought I'd ask though 🙈 .
4. All rich references need to be lazy-imported (imported within the scope that they are used). Either that or the entire module has to be lazy loaded from elsewhere in Cyclopts. We want the typical Cyclopts execution (executing a command, not --help) to be as fast as possible. So that means reducing imports that aren't necessary for that flow (namely rich).

TL;DR I think it's great, please continue the implementation! Probably break up all the code into a few files. Thank you for your work!

[ESPR3SS0]

Glad to hear you like it! Thanks for the feedback, I'll get working on the imports and the organization!

As for your questions:

1. My idea was to have the all column members be dynamic and easy to add, which is why I went for a dict. I wanted the user to be able to create custom entries easily, and to generalize the structure of the table as much as possible. Looking at it now, this probably is better done by having the user inherit the AbstractTableEntry object and adding members, or doing what you said and having AbstractTableEntry.data be a dataclass itself.

I'll tinker with an adjusted implementation. Below I compare what I have in mind now, to how it would currently be used...

## Custom Members as a Dataclass (new implementation idea) ...
class MyCustomMembers(AbstractEntryData):
      will_reset: bool

foo = AbstractTableEntry(name='com1', data=MyCustomMembers(False))
bar = AbstractTableEntry(name='com2', data=MyCustomMembers(True))

## VERSUS 
## Custom Members as Dict Keys (the current implemetation)...

foo = AbstractTableEntry(name='com1')
foo.put('will_reset') = False

bar = AbstractTableEntryname(name='com2')
bar.put('will_reset') = True

## Pseudo code to then add the entries, and a custom column to leverage the custom member in the entries...
new_col = ColumnSpec(key='will_reset', converter=exclamation_mark_if_reset)
panel = AbstractRichHelpPanel()
table.add_column(new_col) 
table.append_entries(foo, bar) 
panel.build(table) 

Brief justification for having dynamic members at all... In some of my CLIs it would be nice to have a column similar to the asterisk column, but have it use some other flag with a different meaning. That way I could tag my commands and give the user some quick and easy to see information! Also, it abstract further the information for the table, so it's just very flexible.

2. The ColumnSpec and TableSpec attributes are copied straight from rich, however, I did not include all of the attributes. For example, rich.Table has a row_style, header_style, footer_style, safe_box that I did not include. I only grabbed the attributes I thought were immediately important, but can match them 1:1 if that would be better.

[BrianPugh]

I defer to your judgement on this! Thanks again for working on this!

I only grabbed the attributes I thought were immediately important, but can match them 1:1 if that would be better.

I'm happy with only having the relevant ones, i just wanted to make sure we didn't have things like "show_header": self.display_header or other slight unnecessary differences.

[ESPR3SS0]

Awesome! I've gone the dataclass route, believe I have moved all the rich imports, and have made a "autododecprate/help/" module! Here's the link again: https://github.com/ESPR3SS0/cyclopts, let me know what you think!

And no problem, cyclopts has been a bliss to use so thank you for making it! I came from Typer and haven't looked back.

There are a few edge cases of tests that are failing.

Set 1 of failing cases is due to the console capturing the pytest command itself. Is there a creative way to run the tests to get around this? I was thinking I could consider a way to provide a program name to the app to get around this too if that sounds like an interesting addition. Below is an example ( I got this on main branch in addition to my clone)...

E         - Usage: test_help COMMAND                                                                                  
E         ?            -----                                                                                          
E         + Usage: pytest COMMAND                                                                                     
E         ?        ++   

Set 2 of failing cases occurs when we have multiple names for parameters, I.E) "test_help_print_commands_plus_meta_short" where hostname: Annotated[str, Parameter(name=['--hostname', '-n'], help="Hostname to connect to.")]. The output of the failed test is:

E         - -hostname  -n  Hostname to connect to. [required]                                                         
E         ?           -                                                                                               
E         + -hostname -n  Hostname to connect to. [required]                                                          
E         ?                                                          +  

Internally, when this is parsed, the name field of the AbstractTableEntry is set to "--hostname" and the short field is set to "-n". Then, the default converter behavior is to display: name + " " + short.

I actually haven't found where the original implementation adds that extra space yet 😅. Therefore, is this extra space something that was intentional? I will debug and find the issue if so, otherwise, would you be okay with a new behavior of one less space?

Set 3 is just a single case "test_help_print_combined_parameter_command_group". Specifically:

E         - --value1      [required]                                 │                                                
E         ?         ----                                                                                              
E         + --value1  [required]                                     │                                                
E         ?                                                      ++++...

The [required] flag is appended to the description of the entry object. This is another case where I do not know why exactly the old implementation had the 4 extra spaces and the knew one does not. So again, is this an intentional format, or would a change in behavior be okay?

Anything else you see that needs changes let me know!

[BrianPugh]

I'm so sorry, I meant a cyclopts.help module/folder, not autodecorate; I was getting my wires crossed with work 🙈 . Basically get rid of the autodecorate folder and move the help folder up one. We're not decorating anything!

1. I usually run pytest with python -m pytest; if that resolves that problem let's just leave that for now.
2. Feel free to add/remove whitespace in the tests if they make sense. I don't believe that extra whitespace was intentional.
3. Probably related to (2). This might be coming from the HelpPanel.__rich_console__ method, around this logic:

   commands_width = ceil(console.width * 0.35)
   

   The goal is is for things to look reasonable at various different console widths. This isn't really captured by the unit tests since we just render them all at a fixed 70 columns.

Basically, the overarching goal with the help unit tests is "does the help-page look good?"

[ESPR3SS0]

I was really wonder what was going to be auto-decorated! No problem I moved the folder to cyclopts/help now.

• python -m pytest fixed the issue too, thank you!
• I've adjust the white spaces in such a way all the commands still look good 👍... and all the tests pass 🎉.

Now I'm going to play with some customized panels and see what making them from a users perspective feels like, I'm curious if some helper methods to construct a panel are needed... then once I have some fun examples of custom panels I think it would be good to add some tests for them.

Tying it back to the start of this thread, I'll make a Panel like @jscholes was looking for!

[BrianPugh]

Awesome thanks! Could you also make a draft PR? It'll allow us to more precisely talk about different parts of the new feature's code.

[BrianPugh]

@ESPR3SS0's PR has been merged into the v4-develop branch. While it doesn't directly expose any new functionality, it does properly abstract a lot of the help-formatting to make it easier to control. I'll be playing around with it shortly with this github issue in mind.

[BrianPugh]

this has now been implemented in the v4-develop branch. The help-page can now be completely customized. Feedback would be appreciated!

See https://cyclopts.readthedocs.io/en/v4-develop/help_customization.html for documentation on this new feature.

I'll close this issue once v4 is actually released.

[ESPR3SS0]

Howdy again, this all looks excellent to me! I code looks much much better, and the implementation via help_formatter feels intuitive and clear to me.

I'll play around some in the v4-branch and see if anything stands out enough for me to mention here.

Thank you for all the work you picked up on this @BrianPugh, I'm very excited to starting using these features.

... now I just may be able to carve some time to work on #180 if you think the HelpPage stuff is settled enough to do so.

[BrianPugh]

That would be super helpful! yes I think the general structure is settled enough that we can iterate on that feature.

[jscholes]

@BrianPugh I haven't had a chance to take this for a spin yet. But I want to thank yourself and @ESPR3SS0 for not only implementing exactly what I asked for, but making the library more flexible for everyone in the process.

Looking forward to trying it out!

[BrianPugh]

v4 beta is out now! Please give it a try and report back and success/issues. I'll let this sit for a week; if feedback is mostly positive I'll perform the full release.

• Changelog: https://github.com/BrianPugh/cyclopts/releases/tag/v4.0.0b1
• Documentation: https://cyclopts.readthedocs.io/en/latest/
• Install it via:

  pip install cyclopts==4.0.0b1
  

[BrianPugh]

v4.0.0 is now released containing this feature/fix.