google-batch disk-type error

[rivershah]

Trying to use n4-standard-* machines. These require hyperdisk-balanced. This is being set in command line via: --disk-type hyperdisk-balanced however job fails with:

Job gets non-retryable information Batch Error: code - CODE_GCE_INVALID_FIELD_VALUE, description - Invalid value for field 'operation': ''. No zone supports all of the provided instance templates. The following errors detail the failure. [pd-standard, n4-standard-2] features are not compatible for creating instance..

[rivershah]

I noticed that if we allow google batch to automatically pick, then the issue goes away. Specifying disk types for newer machines such as n4-standard or a3-highgpu breaks otherwise

diff --git a/dsub/providers/google_batch_operations.py b/dsub/providers/google_batch_operations.py
index 5bba01d..3375edf 100644
--- a/dsub/providers/google_batch_operations.py
+++ b/dsub/providers/google_batch_operations.py
@@ -364,7 +364,7 @@ def build_persistent_disk(
     size_gb: int, disk_type: str
 ) -> batch_v1.types.AllocationPolicy.Disk:
   disk = batch_v1.AllocationPolicy.Disk()
-  disk.type = disk_type
+  # disk.type = disk_type
   disk.size_gb = size_gb
   return disk

[wnojopra]

Hi @rivershah !

From providers README:

--disk-type:
The disk type to use for the data disk. Valid values are pd-standard, pd-ssd and local-ssd. (default: pd-standard)

We'll look into adding support for other disk types - it should be a simple passthrough.