Sudden error with no change of code, data or sdk versions - Failed to upload outputs due to: Failure in UpdateRun while calling service RunHistory

[edgBR]

Bug description

A hyperdrive step which previously was working without any issue has started to fail. There has been no change of code, no change of data, no change of azureml sdk version, no change of anything.

Error is as follows:

Failed to upload outputs due to: Failure in UpdateRun while calling service RunHistory; HttpMethod: PATCH; Response StatusCode: BadRequest; Exception type: Microsoft.RelInfra.Extensions.HttpRequestDetailException|-Microsoft.RelInfra.Common.Exceptions.ErrorResponseException

The hyperdrive job works well:

And the trial worked:

I suspect that maybe the problem is related with the high amount of input datasets I have as I add to implement a workaround for this #26329

Any help is welcome.

[azure-sdk]

Label prediction was below confidence level 0.6 for Model:ServiceLabels: 'Data Factory:0.11201263,KeyVault:0.06103551,Storage:0.06010549'

[mccoyp]

Hi @edgBR, thank you for opening an issue (and providing extra context with a linked issue)! I'll tag some folks who should be able to help, and we'll get back to you as soon as possible.

[ghost]

Thank you for your feedback. This has been routed to the support team for assistance.

[mccoyp]

Tagging @azureml-github for awareness.

[luigiw]

@wangchao1230 can you help to triage this issue? Thank you.

[wangchao1230]

@edgBR Does this issue constantly repo? It may be related to some transient server issue at that time.

Could you share the pipeline run id/name to us?

[ghost]

Hi, we're sending this friendly reminder because we haven't heard back from you in a while. We need more information about this issue to help address it. Please be sure to give us your input within the next 7 days. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!