Docs: describe how key flags work

[jinnatar]

Currently docs say:

--help: prints a list of all key flags (see below).

However, this is the only mention of key flags. Reading issues I see these are inferred automagically from the main module, and that perhaps flags.declare_key_flag can somehow be used to manually declare them, at least in some circumstances. None of this is documented though and scrambling through what little there is, I was not able to make it work in a multi module environment where I want to selectively surface some flags from various modules into the top level --helpshort. As an alternative I could centralize flag definitions, but then I would still need documentation on key flags to be able to select from that central definition which are key flags and which are not.

[jinnatar]

The issue I'm trying to fix is getting flags as key flags from modules in a modular program. Why I run into this even in simple scenarios though is because the default magic breaks with project.scripts declared cli endpoints since they work by injecting an intermediate __main__ as implemented by setuptools & friends. My hope is that by declaring flags as keyflags I can manually define what I want instead of relying on the magic that's failing me. I'll keep investigating if I can somehow force an environment where absl is presented with a world where the magic works.

What follows is a simplified example. While the modularity doesn't necessarily make sense here, it does in a more complex scenario.

%> tree
.
├── pyproject.toml
├── src
│   └── myprog
│       ├── __init__.py
│       └── __main__.py
└── uv.lock

pyproject.toml:

[tool.pdm.build]
includes = []
[build-system]
requires = ["pdm-backend"]
build-backend = "pdm.backend"

[project]
requires-python = "<4.0,>3.9.1"
dependencies = [
    "absl-py<3.0.0,>=2.1.0",
]
name = "myprog"
version = "0.0.1"
description = "absl-py demo myprog"

[project.scripts]
myprog = "myprog.__main__:run"

src/myprog/__main__.py:

from absl import flags, app

FLAGS = flags.FLAGS

flags.DEFINE_boolean("auth", False, "Perform authentication init and nothing else.")
flags.declare_key_flag("auth")  # This seems to have zero effect

def main(argv) -> None:
    del argv
    if FLAGS.auth:
        print("Yes!")
    else:
        print("No!")

def run():
    app.run(main)

run()

This works as expected, the magic works:

%> uv run python -m myprog --helpshort

       USAGE: /home/user/src/myprog.git/src/myprog/__main__.py [flags]
flags:

/home/user/src/myprog.git/src/myprog/__main__.py:
  --[no]auth: Perform authentication init and nothing else.
    (default: 'false')

Try --helpfull to get a list of all flags.

Utilizing the script endpoint it fails:

%> uv run kinobot --helpshort

       USAGE: /home/user/src/myprog.git/.venv/bin/myprog [flags]

Try --helpfull to get a list of all flags.

.. And same result from a built & pipx installed wheel:

%> myprog --helpshort

       USAGE: /home/user/.local/bin/myprog [flags]

Try --helpfull to get a list of all flags.

The automatic script endpoints created are fairly simple, but become the __main__ which breaks the magic on the absl side.

A --helpfull in the 2 broken scenarios lists the flag as:

myprog.__main__:
  --[no]auth: Perform authentication init and nothing else.
    (default: 'false')

uv version of the injected script runner:

#!/home/user/src/myprog.git/.venv/bin/python3
# -*- coding: utf-8 -*-
import sys
from myprog.__main__ import run
if __name__ == "__main__":
    if sys.argv[0].endswith("-script.pyw"):
        sys.argv[0] = sys.argv[0][:-11]
    elif sys.argv[0].endswith(".exe"):
        sys.argv[0] = sys.argv[0][:-4]
    sys.exit(run())

Installed wheel version of the script runner:

#!/home/user/.local/pipx/venvs/myprog/bin/python
# -*- coding: utf-8 -*-
import re
import sys
from myprog.__main__ import run
if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
    sys.exit(run())

[jinnatar]

Reading code others have written declaring key flags I'm forming an educated guess:

• Declaring a flag as a key flag will elevate it's visibility into the scope from which the declaration is made.
• Ergo, it cannot be used to control what is key for an invocation of a script, unless declared directly in that script.
• Ergo, because there doesn't seem to be any way of affecting the python core standard script entrypoints, it cannot be used to fix the underlying issue since we can't inject the key declarations into the shim.
• Ergo, there are two issues here, 1) key flags and their love & care is completely undocumented. 2) absl-py cannot be used to create a modern entrypoint script where --help(short) is capable of showing the most relevant top level flags.

If my chain of logic here is correct and someone can confirm that, I'll split the second issue into a new one.