Prevent buildtest from hanging due to directory traversal

[shahzebsiddiqui]

At the moment, buildtest has support for directory traversal when searching for buildspecs that can lead to serious issues if one specifies a directory path with lots of files. This can lead to a full directory traversal of filesystem which can impact some of the features in buildtest that may include the following options

• buildtest build -b
• buildtest buildspec validate -b
• buildtest buildspec find -d

buildtest uses the walk_tree method using the file_traversal_limit to stop buildtest from searching for files beyond the limit

https://github.com/buildtesters/buildtest/blob/26ed3478b92bd40cc7cd5a8f71ab7125a3bb1c07/buildtest/utils/file.py#L228-L229.

The default value is set to 999999 however this may need to be configurable such that user can alter behavior for directory traversal.

We have a few options we can try to customize this value

1. Add support in configuration file and/or command line that can be overridden. Perhaps a key file_traversal_limit in buildspec cache and option --file-traversal-limit.

In configuration file it would be under the system definition it could be something like this

system:
  generic:
      file_traversal_limit: 5000

The objective is to prevent user input to lead into long directory traversal where one inputs the following. The following walk_tree methods should be updated with value of file_traversla

1. buildtest build -b /tmp

https://github.com/buildtesters/buildtest/blob/0804a13b31491f3657a0444b64e8c19b89ab3525/buildtest/cli/build.py#L550

2. buildtest buildspec validate -b /tmp and buildtest bc find -d /

https://github.com/buildtesters/buildtest/blob/0804a13b31491f3657a0444b64e8c19b89ab3525/buildtest/cli/buildspec.py#L239

For the option we should have it as a global option.

We should not update the walk_tree method with file_traversal_limit in the following invocation

1. This invocation is for buildtest schema

https://github.com/buildtesters/buildtest/blob/0804a13b31491f3657a0444b64e8c19b89ab3525/buildtest/cli/schema.py#L48

4. This invocation is for buildtest history https://github.com/buildtesters/buildtest/blob/0804a13b31491f3657a0444b64e8c19b89ab3525/buildtest/cli/history.py#L113

TODO

• [x] Update the main buildtest configuration file https://github.com/buildtesters/buildtest/blob/devel/buildtest/settings/config.yml and comment it out as default

*Use these in the configuration

# This configuration is used to limit the number of files that can be traversed when searching for buildspecs
# file_traversal_limit: 1000

• [x] Update json schema https://github.com/buildtesters/buildtest/blob/devel/buildtest/schemas/settings.schema.json

• [ ] Update all example configuration files to be consistent everywhere https://github.com/buildtesters/buildtest/tree/devel/buildtest/settings, https://github.com/buildtesters/buildtest/tree/devel/tests/cli/configuration, https://github.com/buildtesters/buildtest/tree/devel/tests/settings, https://github.com/buildtesters/buildtest/tree/devel/tests/cli/cdash_examples https://github.com/buildtesters/buildtest/tree/devel/buildtest/schemas/examples/settings.schema.json/valid

• [ ] Add tests for buildtest build in https://github.com/buildtesters/buildtest/blob/devel/tests/cli/test_build.py

• [ ] Add tests for buildtest buildspec find in https://github.com/buildtesters/buildtest/blob/devel/tests/cli/test_buildspec.py

• [ ] Add implementation for by passing the value this should work for buildtest build, buildtest buildspec find and buildtest buildspec validate , see issue above

*For testing we should try testing the following commands as examples and use file_traversal_limit: 1. You would need to create an example configuration file for testing purposes, put it in https://github.com/buildtesters/buildtest/tree/devel/tests/cli/configuration and use it when testing this feature *

1. buildtest build -b tutorials/
2. buildtest buildspec find -s general_tests
3. buildtest buildspec validate -b tutorials
4. Should try edge cases such as 0 or negative value or a floating point value