AzureML - v1 ParallelRunStep arguments in v2

[timgabrhel]

In the V1 SDK, I can create a ParallelRunStep and execute this directly within a Pipeline (Job, here in v2). The ParallelRunStep allows me to configure a set of arguments that are passed in to the host driver that kicks off each of the steps.

        arguments = ["--model_name", self.model_name,"--first_task_creation_timeout",518400, "--batch_size", self.batch_size]

        batch_score_step = ParallelRunStep(
            name=parallel_step_name,
            inputs=[input_images.as_named_input("input_images")],
            output=output_dir,
            arguments=arguments,
            parallel_run_config=parallel_run_config,
            allow_reuse=False,
        )

This is a snippet from the logging output from 70_driver_log.txt from a completed inference run. You can see the arguments above were passed directly inline to the execution.

Preparing to call script [driver/amlbi_main.py] with arguments:[{redacted}, '--first_task_creation_timeout', '518400', '--batch_size', '20']

My specific goal is to be able to specify the first_task_creation_timeout parameter while using the v2 SDK. I am processing a large dataset and the error message received by AzureML upon execution timeout was to use this parameter.

How can I perform this in v2? I don't see any arguments, or detailed enough documentation for other properties, that would allow me to configure. I have looked in depth using the HTTP REST API as well as the V2 Python SDK, and none of these entities offer support:

• BatchEndpoints
• BatchDeployments
• Jobs

[azure-sdk]

Label prediction was below confidence level 0.6 for Model:ServiceLabels: 'Batch:0.092903234,Storage:0.05441652,Cosmos:0.049568687' Label prediction was below confidence level 0.6 for Model:CategoryLabels: 'Client:0.57247156,Mgmt:0.4217827,Service:0.005308377'

[ghost]

Thank you for your feedback. This has been routed to the support team for assistance.

[xiangyan99]

@azureml-github

[bupt-wenxiaole]

@timgabrhel please reference https://github.com/Azure/azureml-examples/blob/sdk-preview/sdk/jobs/pipelines/1g_pipeline_with_parallel_nodes/pipeline_with_parallel_nodes.ipynb for how to set the arguments. you may need the extra permission to access this notebook. note that the PRS SDK V2 is under the private preview, and the argument name has been changed from "arguments" to "args",

[ghost]

Hi @timgabrhel. Thank you for opening this issue and giving us the opportunity to assist. We believe that this has been addressed. If you feel that further discussion is needed, please add a comment with the text “/unresolve” to remove the “issue-addressed” label and continue the conversation.

[ghost]

Hi @timgabrhel, since you haven’t asked that we “/unresolve” the issue, we’ll close this out. If you believe further discussion is needed, please add a comment “/unresolve” to reopen the issue.