sagemaker-debugger
sagemaker-debugger copied to clipboard
Provide a way to randomly access a step
Current impl : If there are n steps present, all n steps index would be downloaded before the call finishes.
We should allow a way to randomly access step. Use case:
- I want the tensor from last step. or I want the tensor from step 5000-8000
In both these cases, user is only interested in certain steps rather than all steps while doing analysis.
Estimation of time - If there are 50K saved , it will take 50 list calls + 50K get calls( which should be in ~ 20 sec according to benchmark https://github.com/awslabs/sagemaker-debugger/pull/80 where 10K files take 4 sec) to get the index file. 50 list calls can be expensive as those are sequential operation and list is expensive operation.