U-Boot integration [WIP]

[sjg20]

Cover the drivers, scripts, pytest and the Gitlab information.

Note: This implementation is the result of working through what is needed in Labgrid to support U-Boot labs. I did initially file quite a few issues[1] but there has not been a lot of traction and I got feedback that I have overwhelmed people with too many. So I have stopped filing issues on the things I hit along the way. I have carried on with the implementation in the hope that this can be a better basis for discussion.

Note that all of these patches are work-in-progress, but it is becoming more stable.

Feedback may still change the implementation and documentation but the time for wholesale changes of approach has passed.

Cover-letter: U-Boot integration END

Changes in v7:

• Collect all info before printing anything
• Make the query synchronous
• Drop the extra argument from print_version()
• Add new patch to support using kvm
• Improve error message when servod fails
• Add SCP support
• Add OpenSBI support
• Add Rockchip TPL support
• Add support for using a custom build target
• Add support for copying files intot he build
• Add support for using a custom devicetree
• Add hack to deal with colibri-mx8's broken build
• Avoid changing the program environment
• Add support for rpi5, zynqmp, riscv, mx8
• Correct the step title and exception for send()
• Support Jetson Nano as well
• Add new patch to allow setting the monitor-path with UUU
• Correct 'take' typo in subject
• Add new patch to detect read-only devices
• Add patch to simplify the top-level logic
• Drop comment about console not appearing
• Allow recovery to be missing for the board
• Add patch to support boards which need power for console
• Add patch to support boards which share power and reset
• Add support for boards which cannot power on with reset
• Add patch to support boards which need a button to power on
• Use LABGRID_SJG environment variable install of LABGRID
• Set HOSTNAME variable and use it in ub-pyt
• Rename lab config from env.cfg to kea_env.cfg

Changes in v6:

• Show a more friendly error when the wrong query arguments are given
• Add new patch to report an error if QEMU is not turned on
• Improve handling of dual-build targets so pytest works correctly
• Support QEMU on ARM
• Support writing a VBE (Verified Boot for Embedded) TPL file
• Fix reference to crossbar in _ub-bisect-try and ub-smoke
• Add -x option to specify the extra board directory
• Add comments to the example lg-env
• Define USE_LABGRID to tell u-boot-hooks to use labgrid hooks
• Add a few more comments to ub-pyt
• Show the build path with ub-pyt -v
• Add --allow-unmatched to ub-smoke to avoid QEMU error
• Add a patch with an example lab

Changes in v4:

• Get internal console working with qemu
• Show a prompt when starting, to indicate it is waiting for the board

Simon Glass (62): util/helper: Allow setting environment and cwd util/helper: Change command output level to debug helper: Support long-running processes Provide variables to control drivers from the command line driver: Add driver for Dediprog SPI-flash emulator protocol: Add a new recovery protocol protocol: Allow more control of reset labgrid: Add a way to query information from a driver remote/client: Support finding a place by role driver/qemudriver: Make extra_args, cpu and machine optional driver/qemudriver: Delay setting of the QEMU arguments driver/qemudriver: Allow the BIOS filename to be changed driver/qemudriver: Report an error if QEMU is not turned on driver: Add a digital-output driver for recovery driver: Add a driver for the Servo board driver: Add a way to build U-Boot images driver: Provide a driver for writing U-Boot to a board Add a driver for a device which is always powered Correct binding for HIDRelayDriver bootstrapprotocol: Support a boot phase driver/usbloader: Add a USB loader for sunxi / Allwinner driver/usbloader: Add a loader for Tegra driver/usbloader: Add a loader for samsung driver/usbstoragedriver: Allow block size in write_image() driver/usbstoragedriver: Allow block count in write_image() driver/usbstoragedriver: Adjust how sync works driver/usbstoragedriver: Write silently driver/usbstoragedriver: Report time take to write pytestplugin: Allow selecting a particular target pytestplugin: Allow setting variables usbhidrelay: Support generating a recovery signal remote/exporter: Reduce the verbosity remote/exporter: Reduce the verbosity further remote/exporter: Indicate when the exporter is ready remote/client: Provide an option to acquire a place remote/client: Using logging for progress output remote/client: Show the result of a failed command remote/client: Allow releasing a target silently remote/client: Fix environemnt typo remote/client: Use logging when acquiring / releasing remote/client: Flush the console device only when needed remote/client: Move initial-state code into a function remote/client: Provide a way to set the strategy end-state remote/client: Allow checking if the place is acquired remote/client: Move terminal handling into a separate file remote/client: Provide an internal console ssh: Avoid output when running rsync target: Add documentation target: Add documentation for get_driver() target: Allow looking for an optional driver ubootdriver: Support collecting the version string ubootdriver: Provide a way to collect output strategy: Show the U-Boot version when booting strategy: Add a new state to start U-Boot strategy: Allow a board to be bootstrapped strategy: Allow sending U-Boot over USB strategy: Show console output on failure strategy: Support boards which need a recovery button strategy: Support lab mode contrib: Provide some useful scripts for U-Boot Add an example lab doc: Add documentation for U-Boot integration [v2]

contrib/u-boot/.gitignore | 1 + contrib/u-boot/_ub-bisect-try | 47 + contrib/u-boot/conftest.py | 21 + contrib/u-boot/example_env.cfg | 1245 +++++++++++++++++++++++++ contrib/u-boot/example_export.yaml | 938 +++++++++++++++++++ contrib/u-boot/example_places.yaml | 152 +++ contrib/u-boot/get_args.sh | 136 +++ contrib/u-boot/index.rst | 232 +++++ contrib/u-boot/lg-client | 11 + contrib/u-boot/lg-env | 24 + contrib/u-boot/test_smoke.py | 3 + contrib/u-boot/ub-bisect | 45 + contrib/u-boot/ub-cli | 40 + contrib/u-boot/ub-int | 42 + contrib/u-boot/ub-pyt | 74 ++ contrib/u-boot/ub-smoke | 49 + doc/configuration.rst | 611 +++++++++++- doc/usage.rst | 401 ++++++++ labgrid/driver/init.py | 10 +- labgrid/driver/common.py | 11 + labgrid/driver/consoleexpectmixin.py | 7 + labgrid/driver/powerdriver.py | 29 + labgrid/driver/qemudriver.py | 86 +- labgrid/driver/recoverydriver.py | 25 + labgrid/driver/resetdriver.py | 7 + labgrid/driver/servodriver.py | 157 ++++ labgrid/driver/sfemulatordriver.py | 102 ++ labgrid/driver/ubootdriver.py | 28 +- labgrid/driver/ubootproviderdriver.py | 326 +++++++ labgrid/driver/ubootwriterdriver.py | 169 ++++ labgrid/driver/usbhidrelay.py | 7 +- labgrid/driver/usbloader.py | 170 +++- labgrid/driver/usbstoragedriver.py | 18 +- labgrid/factory.py | 4 +- labgrid/protocol/init.py | 1 + labgrid/protocol/bootstrapprotocol.py | 8 +- labgrid/protocol/recoveryprotocol.py | 14 + labgrid/protocol/resetprotocol.py | 12 + labgrid/pytestplugin/fixtures.py | 21 +- labgrid/pytestplugin/hooks.py | 8 + labgrid/remote/client.py | 266 ++++-- labgrid/remote/config.py | 7 +- labgrid/remote/exporter.py | 123 ++- labgrid/resource/init.py | 6 + labgrid/resource/remote.py | 24 + labgrid/resource/servo.py | 485 ++++++++++ labgrid/resource/sfemulator.py | 32 + labgrid/resource/suggest.py | 6 + labgrid/resource/udev.py | 33 + labgrid/strategy/ubootstrategy.py | 148 ++- labgrid/target.py | 120 ++- labgrid/util/helper.py | 222 +++-- labgrid/util/ssh.py | 3 +- labgrid/util/term.py | 196 ++++ labgrid/var_dict.py | 8 + man/labgrid-client.1 | 6 + man/labgrid-client.rst | 4 + man/labgrid-device-config.5 | 4 + 58 files changed, 6715 insertions(+), 270 deletions(-) create mode 100644 contrib/u-boot/.gitignore create mode 100755 contrib/u-boot/_ub-bisect-try create mode 100644 contrib/u-boot/conftest.py create mode 100644 contrib/u-boot/example_env.cfg create mode 100644 contrib/u-boot/example_export.yaml create mode 100644 contrib/u-boot/example_places.yaml create mode 100755 contrib/u-boot/get_args.sh create mode 100644 contrib/u-boot/index.rst create mode 100755 contrib/u-boot/lg-client create mode 100755 contrib/u-boot/lg-env create mode 100644 contrib/u-boot/test_smoke.py create mode 100755 contrib/u-boot/ub-bisect create mode 100755 contrib/u-boot/ub-cli create mode 100755 contrib/u-boot/ub-int create mode 100755 contrib/u-boot/ub-pyt create mode 100755 contrib/u-boot/ub-smoke create mode 100644 labgrid/driver/recoverydriver.py create mode 100644 labgrid/driver/servodriver.py create mode 100644 labgrid/driver/sfemulatordriver.py create mode 100644 labgrid/driver/ubootproviderdriver.py create mode 100644 labgrid/driver/ubootwriterdriver.py create mode 100644 labgrid/protocol/recoveryprotocol.py create mode 100644 labgrid/resource/servo.py create mode 100644 labgrid/resource/sfemulator.py create mode 100644 labgrid/util/term.py create mode 100644 labgrid/var_dict.py

[sjg20]

Changes for v3:

• Rebase on top of grpc branch
• Don't mess with terminal setting unless stdin is a terminal
• Don't show an error if there are no resources when auto-acquiring
• Support QEMU in UBootWriter

[sjg20]

Changes in v4:

• Support for Beagleplay, which needs files from two separate U-Boot builds
• Support for a 'recovery' button needed to boot the image
• Tidy up the internal-console support
• Fix pytest behaviour with an unpatched U-Boot (that doesn't have lab mode)