Explain cache key for other types of steps

[shinglyu]

What did you find confusing? Please describe. I try to understand which SageMaker SDK parameter would cause my SageMaker Pipeline step to re-run, this documentation only gives an example of the processing step. We need documentation on the other steps like training, clarify etc.

The developer documentation list these cache keys using low-level AWS SDK terminology, which cannot be easily mapped to SageMaker SDK paramters without extensive code reading.

Describe how documentation can be improved For each pipeline step type that has caching, explain which paramter change would cause a re-run.

Additional context Add any other context or screenshots about the documentation request here.

[brockwade633]

Hi @shinglyu, both processing and training step types are included in the SDK docs when describing what SDK parameters will cause cache misses and step re-executions. These are the most heavily used by customers and so we prioritized specific examples for those.

You are right though, for the remaining step types, a translation is required from the SageMaker API Step parameters to the Python SDK objects for the reader in order to discern which PySDK params will cause cache hits / misses when changed. I agree it would be nice to include a table of some sort that lists all the params by step type so the customer doesn't have to make that translation, I can add this to our backlog. In the meantime, is there any specific step / situation I can help with?

[francisco-camargo]

I have spent a lot of time trying to get caching to work. It works well and I use it daily for all steps except for TransformStep and Clarify steps for which I have never been able to get caching to work. As Shing has mentioned, more documentation with examples would be very helpful.