Machine type with name 'custom-124-819200' does not exist

[sansarsecondary]

A high memory job is always erroring out. I can replicate with the following simple command::

dsub \
--provider google-cls-v2 \
--project xxx \
--regions us-central1 \
--logging xxx\
--output OUT=xxx\
--command 'echo "Hello World" > "${OUT}"' \
--wait --disk-size 100 \
--image xxx \
--min-ram 800 --min-cores 32

["Error in xxx - code 3: Execution failed: allocating: creating instance: inserting instance: Invalid value for field 'resource.machineType': 'zones/us-central1-a/machineTypes/custom-124-819200'. Custom Machine type with name 'custom-124-819200' does not exist."]
JobExecutionError: One or more jobs finished with status FAILURE or CANCELED during wait.

I could specify machine type:

--machine-type n2-highmem-128

That being said, prefer to have the custom sizing work as it is the most flexible and less user code needed. Thanks!

[wnojopra]

Hi @sansarsecondary !

When support for custom machine types was added, the pipelines api only supported N1 machine types. Support for N2 machine types was being worked on (I see Matt mention it in this comment). N1 machine types have a limit of 624 GB of ram for a single machine.

Since then, support for n2 machines have been added to the API but dsub needs to be updated to figure out that the custom machine type needs to be an n2 machine. This will be added in a future release of dsub.

In the meantime, I'm happy to hear that you're able to specify the machine-type, and would recommend using that until we add support for custom n2 machine types in dsub.