sagemaker-debugger icon indicating copy to clipboard operation
sagemaker-debugger copied to clipboard

Tensors not saved in PREDICT step

Open NihalHarish opened this issue 5 years ago • 1 comments

if self._prepared_tensors[mode]:
            if self._exported_collections is False:
                # in keras, these collections change when mode changes
                # but rest of the project isn't yet capable of handling this
                # this means that collections like outputs, or other collections with intermediate tensors
                # will only have tensor names from first mode

                # this means sometimes collections will be exported after 1 step
                self.export_collections()
                self._exported_collections = True

If we run the same mode consecutively, we end up exporting collections before all the modes are run.

NihalHarish avatar Jun 03 '20 18:06 NihalHarish

Can you provide how to reproduce ? Output and expected result.

Vikas-kum avatar Jun 04 '20 16:06 Vikas-kum