thin-edge.io icon indicating copy to clipboard operation
thin-edge.io copied to clipboard

Create tasks that will follow the Documentation instructions

Open gligorisaev opened this issue 9 months ago • 5 comments

Proposed changes

For documentation checking purposes I am creating collection of End to End tasks which will follow the descriptions/instructions from thinedge documentation. These tasks are running on an Raspberry PI. The idea behind is that as soon as some task fail that would mean that changes in the documentations are needed or some regression mailfunction has been detected.

Types of changes

  • [ ] Bugfix (non-breaking change which fixes an issue)
  • [x] New feature (non-breaking change which adds functionality)
  • [ ] Improvement (general improvements like code refactoring that doesn't explicitly fix a bug or add any new functionality)
  • [ ] Documentation Update (if none of the other choices apply)
  • [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Paste Link to the issue


Checklist

  • [x] I have read the CONTRIBUTING doc
  • [x] I have signed the CLA (in all commits with git commit -s)
  • [ ] I ran cargo fmt as mentioned in CODING_GUIDELINES
  • [ ] I used cargo clippy as mentioned in CODING_GUIDELINES
  • [ ] I have added tests that prove my fix is effective or that my feature works
  • [ ] I have added necessary documentation (if appropriate)

Further comments

gligorisaev avatar May 15 '24 08:05 gligorisaev

Robot Results

:white_check_mark: Passed :x: Failed :next_track_button: Skipped Total Pass % :stopwatch: Duration
433 0 3 433 100 58m56.358857s

github-actions[bot] avatar May 15 '24 08:05 github-actions[bot]

For now no additional tests (tasks) are planned for this PR

gligorisaev avatar Jul 22 '24 09:07 gligorisaev

Some questions:

  • I see that in RobotFramework besides tests one can create tasks. How are they different and why are we doing it as tasks and not tests in this case?
  • Tasks are used for automating processes that are not directly related to testing but rather to operations, administrative tasks, or other kinds of workflows.
  • Tests are designed specifically for validating the functionality, performance, and reliability of software applications. They include assertions to verify that the application behaves as expected.
  • As far as I can see these tasks are not part of the normal test suite (in the RobotFramework/tests/ directory) and they're not run using a docker adapter, but physical devices. In this case will running these tasks be automated to happen on PRs, as current integration test suite is, or will it only be run manually?
  • The purpose of this tasks is to keep the integrity and correctness of our documentation under controll, meaning, if task fails than we need to adjust the documentation
  • Trying to run this task suite on my end, I am getting some errors because tasks assume a home directory of /home/pi. Is the goal of these tasks to run them under a controlled environment so these assumptions ultimately don't matter, or should we try to keep it portable? IMO it would be nice to run them on the developer machines, but perhaps that's not possible due to some blockers?
  • Running these task understand to adjust your .env file like:

SSH - Device Target

SSH_CONFIG_HOSTNAME=192.168.193.56 SSH_CONFIG_USERNAME=pi SSH_CONFIG_PASSWORD=thinedge

just add the information of your developer machine (or like in the example use one of the devices available on ZeroTier)

gligorisaev avatar Jul 23 '24 08:07 gligorisaev

  • Trying to run this task suite on my end, I am getting some errors because tasks assume a home directory of /home/pi. Is the goal of these tasks to run them under a controlled environment so these assumptions ultimately don't matter, or should we try to keep it portable? IMO it would be nice to run them on the developer machines, but perhaps that's not possible due to some blockers?

Though we should avoid using the /home/pi directory explicitly as this is not a standard linux user, and would only work on devices using a raspberry pi built OS image.

reubenmiller avatar Jul 23 '24 09:07 reubenmiller

@gligorisaev What is the state of this PR, it seems to be still a work-in-progress. In that case it would be better to set it to the Draft state. It is unclear which comments have been addressed or not as there aren't any links to the commits which address the comments.

reubenmiller avatar Oct 21 '24 12:10 reubenmiller