Monitor metric calculation pipeline failing with custom preprocessing component

[adfi]

• Package Name: azure-ml-ai
• Package Version: 1.11.0
• Operating System: Ubuntu
• Python Version: 3.10

Describe the bug I am trying to get a Monitor up and running for the case where I have my own production data. I have followed all the instructions and comply with the prerequisites. When running the python code from the instructions the monitor is created just fine. However when executing the pipeline for the metric calculation the Preprocessing Component fails with the following error: Failed to submit job due to Exception: Response status code does not indicate success: 400 (Must specify ComputeType in the request when submit job to Global Job Dispatcher.). Must specify ComputeType in the request when submit job to Global Job Dispatcher..

The other components in the pipeline all have the serverless spark compute assigned to it but the Custom Preprocessing component does not.

To Reproduce Steps to reproduce the behavior:

1. Create a custom preprocessing component
2. Follow the instructions to create a monitor
3. Manually trigger the pipeline to calculate the metrics

Expected behavior I expected the pipeline to complete successfully.

[github-actions[bot]]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @azureml-github @Azure/azure-ml-sdk.

[annatisch]

Thanks for the report @adfi! I have looped in the ML team to take a look :)

[mats-claassen]

Hi, I am facing the same problem. Is there a workaround or solution?

[anaarmas-sys0]

Hi guys! Im facing the same problem, Do you have any suggestion please?

[anaarmas-sys0]

Here the issue

[anaarmas-sys]

Hi again!! Any suggestion until now?

[ahughes-msft]

Hello @adfi, @mats-claassen, @anaarmas-sys0

For creating the custom preprocessing component to preprocess your data, please see the examples here: https://github.com/Azure/azureml-examples/tree/main/cli/monitoring/components/custom_preprocessing

I would like to understand more about the challenges your are experiencing here and the format of the data you are attempting to preprocess. If you could please share a contact email I will reach out to you to schedule a call.

Many thanks, Alex

[anaarmas-sys0]

Hello Alex!

For sure!, my email is @.*** In the meantime I'm going to check the example in the link.

Thank you!

El mié, 10 ene 2024 a las 13:15, Alexander Hughes @.***>) escribió:

Hello @adfi https://github.com/adfi,

For creating the custom preprocessing component to preprocess your data, please see the examples here: https://github.com/Azure/azureml-examples/tree/main/cli/monitoring/components/custom_preprocessing

I would like to understand more about the challenges your are experiencing here and the format of the data you are attempting to preprocess. If you could please share a contact email I will reach out to you to schedule a call.

Many thanks, Alex

— Reply to this email directly, view it on GitHub https://github.com/Azure/azure-sdk-for-python/issues/32427#issuecomment-1885504649, or unsubscribe https://github.com/notifications/unsubscribe-auth/BEMSH7SA4IDBV6EYMHQCSADYN3SDVAVCNFSM6AAAAAA53USLIOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBVGUYDINRUHE . You are receiving this because you commented.Message ID: @.***>

[anaarmas-sys]

Hello Alex! The example in the link solved the problem about the ComputeType requested. Thank you so much!

[adfi]

The matter isn't relevant for me anymore. As @anaarmas-sys reports it to be resolved, we can close the issue.