openfl icon indicating copy to clipboard operation
openfl copied to clipboard

Modifying the `shard_num` read for templates to indicate that `data_path` flag needs to be passed as an integer

Open theakshaypant opened this issue 1 year ago • 2 comments

Description

This is a documentation patch. Currently, docs seem to show data_path to be an optional flag. But for the specific example, it is mandatory. This PR makes the mention explicit.

In addition, a guardrail to expect data_path as a shard number is added in all affected tutorials.

PR Changes

  • Updated the docs to represent that data_path is not an optional parameter for the tutorial.
  • Adds a detailed error message if the shard_num is not an integer value.
  • This change is made in the following templates:
    • keras_cnn_mnist
    • keras_cnn_with_compression
    • torch_cnn_histology
    • torch_cnn_histology_gramine_ready
    • torch_cnn_mnist
    • torch_cnn_mnist_eden_compression
    • torch_cnn_mnist_fed_eval
    • torch_cnn_mnist_straggler_check
    • torch_unet_kvasir
    • torch_unet_kvasir_gramine_ready

theakshaypant avatar Aug 26 '24 07:08 theakshaypant

I think extracting first integer from the string is a dangerous assumption. From my understanding, those data paths (e.g. data/c1 or data/collab2) are not actual locations on disk (beyond the fact that user may also provide data/100shards/shard0, wherein this regex will fail).

In this case, try casting data_path to int or raise a ValueError that asks user to pass shard ID in the -d <shard_num> format, since data path does not mean anything in this example.

  • Added exception handling when reading shard number from data path in 57809a692f41bd3c78c5eff7d808231254101092.
  • Removed the optional tag with data path in the docs in 70fe34afac99ca3e6035a2e1f433bafd81def28d.

theakshaypant avatar Aug 26 '24 09:08 theakshaypant

@theakshaypant Please rebase your branch with develop. PKI test needs to pass.

MasterSkepticista avatar Aug 27 '24 04:08 MasterSkepticista