azure-xplat-cli Azure vm set new os disk size fails with a serialization issue

CLI Version: 0.10.5 (node: 6.3.1) OS Type: Bash on Ubuntu on Windows 10 version 1607 (OS Build 14392.187) Installation via: npm Mode: ARM

Environment: AzureCloud

Description: With reference to fix #2601 - I tried using the newly available command to change the OS disk size of VM as below:

azure vm set --new-os-disk-size 512 --resource-group ra-single-vm-rg --name ra-single-vm-vm0

Steps to reproduce:

Used the template available here to create a single VM with default OS disk size.
Then run azure vm set --new-os-disk-size 512 --resource-group ra-single-vm-rg --name ra-single-vm-vm0 to reset the size of OS disk to 512.

It looks like an issue with setting the primary property of the NIC. The parameters file supplied with the RA sets the property as string "true" but JSON serialization expects a Boolean format. But changing the parameter to true instead of "true" leads to template failure!

Error stack trace:

2016-10-11T22:06:44.584Z: { Error: Error "parameters.properties.networkProfile.networkInterfaces.properties.primary with value true must be of type boolean." occurred in serializing the payload - "{ id: '/subscriptions/d0d422cd-e446-42aa-a2e2-e88806508d3b/resourceGroups/ra-single-vm-rg/providers/Microsoft.Compute/virtualMachines/ra-single-vm-vm0', name: 'ra-single-vm-vm0', type: 'Microsoft.Compute/virtualMachines', location: 'eastus2', tags: { displayName: 'VM' }, hardwareProfile: { vmSize: 'Standard_DS1_v2' }, storageProfile: { imageReference: { publisher: 'Canonical', offer: 'UbuntuServer', sku: '14.04.5-LTS', version: 'latest' }, osDisk: { osType: 'Linux', name: 'ra-single-vm-vm0-os.vhd', vhd: { uri: 'http://vmhmprtuaxhsyaist1.blob.core.windows.net/vhds/ra-single-vm-vm0-os.vhd' }, caching: 'ReadWrite', createOption: 'FromImage', diskSizeGB: 512 }, dataDisks: [ { lun: 0, name: 'dataDisk1', vhd: { uri: 'http://vmhmprtuaxhsyaist1.blob.core.windows.net/vhds/ra-single-vm-vm0-dataDisk1.vhd' }, caching: 'None', createOption: 'Empty', diskSizeGB: 128 }, { lun: 1, name: 'dataDisk2', vhd: { uri: 'http://vmhmprtuaxhsyaist1.blob.core.windows.net/vhds/ra-single-vm-vm0-dataDisk2.vhd' }, caching: 'None', createOption: 'Empty', diskSizeGB: 128 } ] }, osProfile: { computerName: 'cn0', adminUsername: 'testuser', linuxConfiguration: { disablePasswordAuthentication: false }, secrets: [] }, networkProfile: { networkInterfaces: [ { id: '/subscriptions/d0d422cd-e446-42aa-a2e2-e88806508d3b/resourceGroups/ra-single-vm-rg/providers/Microsoft.Network/networkInterfaces/ra-single-vm-vm0-nic1', primary: 'true' } ] }, diagnosticsProfile: { bootDiagnostics: { enabled: true, storageUri: 'http://vmhmprtuaxhsyaidiag1.blob.core.windows.net' } }, provisioningState: 'Succeeded', vmId: 'bab8e7e4-7c36-4ca1-98f9-de04c6aa484b' }" <<< async stack >>> at VirtualMachines.beginCreateOrUpdate (/usr/lib/node_modules/azure-cli/node_modules/azure-arm-compute/lib/operations/virtualMachines.js:865:30) at VirtualMachines.createOrUpdate (/usr/lib/node_modules/azure-cli/node_modules/azure-arm-compute/lib/operations/virtualMachines.js:531:8) at createOrUpdateVM__14 (/usr/lib/node_modules/azure-cli/lib/commands/arm/vm/virtualMachine.js:208:82) at createOrUpdateVM__14 (/usr/lib/node_modules/azure-cli/lib/commands/arm/vm/virtualMachine.js:208:82) at setVM__20 (/usr/lib/node_modules/azure-cli/lib/commands/arm/vm/vmClient.js:854:20) at __18 (/usr/lib/node_modules/azure-cli/lib/commands/arm/vm/vm.js:415:16) <<< raw stack >>> at VirtualMachines.beginCreateOrUpdate (/usr/lib/node_modules/azure-cli/node_modules/azure-arm-compute/lib/operations/virtualMachines.js:865:30) at VirtualMachines.createOrUpdate (/usr/lib/node_modules/azure-cli/node_modules/azure-arm-compute/lib/operations/virtualMachines.js:531:8) at __$createOrUpdateVM__14 (/usr/lib/node_modules/azure-cli/lib/commands/arm/vm/virtualMachine.js:208:82) at __tryCatch (/usr/lib/node_modules/azure-cli/node_modules/streamline/lib/callbacks/runtime.js:150:4) at ___ (/usr/lib/node_modules/azure-cli/lib/commands/arm/vm/virtualMachine.js:192:118) at ___ (/usr/lib/node_modules/azure-cli/lib/commands/arm/vm/virtualMachine.js:212:58) at __$createOrUpdateVM__14 (/usr/lib/node_modules/azure-cli/lib/commands/arm/vm/virtualMachine.js:214:159) at __func (/usr/lib/node_modules/azure-cli/node_modules/streamline/lib/callbacks/runtime.js:47:5) at VirtualMachine.createOrUpdateVM__14 as createOrUpdateVM at ___ (/usr/lib/node_modules/azure-cli/lib/commands/arm/vm/vmClient.js:854:33) stack: [Getter/Setter], __frame: { name: 'createOrUpdateVM__14', line: 190, file: '/usr/lib/node_modules/azure-cli/lib/commands/arm/vm/virtualMachine.js', prev: { name: 'setVM__20', line: 828, file: '/usr/lib/node_modules/azure-cli/lib/commands/arm/vm/vmClient.js', prev: [Object], calls: 3, active: false, offset: 26, col: 19 }, calls: 0, active: false, offset: 18, col: 81 }, rawStack: [Getter] } Error: Error "parameters.properties.networkProfile.networkInterfaces.properties.primary with value true must be of type boolean." occurred in serializing the payload - "{ id: '/subscriptions/d0d422cd-e446-42aa-a2e2-e88806508d3b/resourceGroups/ra-single-vm-rg/providers/Microsoft.Compute/virtualMachines/ra-single-vm-vm0', name: 'ra-single-vm-vm0', type: 'Microsoft.Compute/virtualMachines', location: 'eastus2', tags: { displayName: 'VM' }, hardwareProfile: { vmSize: 'Standard_DS1_v2' }, storageProfile: { imageReference: { publisher: 'Canonical', offer: 'UbuntuServer', sku: '14.04.5-LTS', version: 'latest' }, osDisk: { osType: 'Linux', name: 'ra-single-vm-vm0-os.vhd', vhd: { uri: 'http://vmhmprtuaxhsyaist1.blob.core.windows.net/vhds/ra-single-vm-vm0-os.vhd' }, caching: 'ReadWrite', createOption: 'FromImage', diskSizeGB: 512 }, dataDisks: [ { lun: 0, name: 'dataDisk1', vhd: { uri: 'http://vmhmprtuaxhsyaist1.blob.core.windows.net/vhds/ra-single-vm-vm0-dataDisk1.vhd' }, caching: 'None', createOption: 'Empty', diskSizeGB: 128 }, { lun: 1, name: 'dataDisk2', vhd: { uri: 'http://vmhmprtuaxhsyaist1.blob.core.windows.net/vhds/ra-single-vm-vm0-dataDisk2.vhd' }, caching: 'None', createOption: 'Empty', diskSizeGB: 128 } ] }, osProfile: { computerName: 'cn0', adminUsername: 'testuser', linuxConfiguration: { disablePasswordAuthentication: false }, secrets: [] }, networkProfile: { networkInterfaces: [ { id: '/subscriptions/d0d422cd-e446-42aa-a2e2-e88806508d3b/resourceGroups/ra-single-vm-rg/providers/Microsoft.Network/networkInterfaces/ra-single-vm-vm0-nic1', primary: 'true' } ] }, diagnosticsProfile: { bootDiagnostics: { enabled: true, storageUri: 'http://vmhmprtuaxhsyaidiag1.blob.core.windows.net' } }, provisioningState: 'Succeeded', vmId: 'bab8e7e4-7c36-4ca1-98f9-de04c6aa484b' }" <<< async stack >>> at VirtualMachines.beginCreateOrUpdate (/usr/lib/node_modules/azure-cli/node_modules/azure-arm-compute/lib/operations/virtualMachines.js:865:30) at VirtualMachines.createOrUpdate (/usr/lib/node_modules/azure-cli/node_modules/azure-arm-compute/lib/operations/virtualMachines.js:531:8) at createOrUpdateVM__14 (/usr/lib/node_modules/azure-cli/lib/commands/arm/vm/virtualMachine.js:208:82) at createOrUpdateVM__14 (/usr/lib/node_modules/azure-cli/lib/commands/arm/vm/virtualMachine.js:208:82) at setVM__20 (/usr/lib/node_modules/azure-cli/lib/commands/arm/vm/vmClient.js:854:20) at __18 (/usr/lib/node_modules/azure-cli/lib/commands/arm/vm/vm.js:415:16) <<< raw stack >>> at VirtualMachines.beginCreateOrUpdate (/usr/lib/node_modules/azure-cli/node_modules/azure-arm-compute/lib/operations/virtualMachines.js:865:30) at VirtualMachines.createOrUpdate (/usr/lib/node_modules/azure-cli/node_modules/azure-arm-compute/lib/operations/virtualMachines.js:531:8) at __$createOrUpdateVM__14 (/usr/lib/node_modules/azure-cli/lib/commands/arm/vm/virtualMachine.js:208:82) at __tryCatch (/usr/lib/node_modules/azure-cli/node_modules/streamline/lib/callbacks/runtime.js:150:4) at ___ (/usr/lib/node_modules/azure-cli/lib/commands/arm/vm/virtualMachine.js:192:118) at ___ (/usr/lib/node_modules/azure-cli/lib/commands/arm/vm/virtualMachine.js:212:58) at __$createOrUpdateVM__14 (/usr/lib/node_modules/azure-cli/lib/commands/arm/vm/virtualMachine.js:214:159) at __func (/usr/lib/node_modules/azure-cli/node_modules/streamline/lib/callbacks/runtime.js:47:5) at VirtualMachine.createOrUpdateVM__14 as createOrUpdateVM at ___ (/usr/lib/node_modules/azure-cli/lib/commands/arm/vm/vmClient.js:854:33)

Oct 11 '16 22:10 kirpasingh

Is it feasible to get the -vv logs for azure vm set?

Oct 12 '16 01:10 huangpf

@amarzavery The problem was caused by various sides:

service side shall not allow string input of 'true' for the primary field, which should be boolean;
Node.js client accepts it from get() but rejects it at createOrUpdate, as shown and implied from the logs above;
VM command does not do a clean up on all boolean input fields, which is too exhaustive to exercise.

Reference in log:

{ networkInterfaces:
[ { id: '/subscriptions/d0d422cd-e446-42aa-a2e2-e88806508d3b/resourceGroups/ra-single-vm-rg/providers/Microsoft.Network/networkInterfaces/ra-single-vm-vm0-nic1',
primary: 'true' } ] },

Oct 12 '16 07:10 huangpf

@amarzavery I can make a temporary fix to 3), but I think the true fix shall be on your team for 2), because there could be numerous cases similar to this.

Oct 12 '16 07:10 huangpf

@huangpf and @amarzavery Thank you guys for looking into this. In light of the fact that equivalent PS cmdlets are able to expand the OS disk (specifically in this particular case where Boolean is passed as string in the parameters interchangeably) without an issue, my 2 cents would be to have it fixed at Node.js level (as @huangpf suggested) given the random scattering of such cases overall.

Oct 12 '16 17:10 kirpasingh

@huangpf - would #1929 fix this? If so, we can close this issue and track the work in 1929

Oct 12 '16 22:10 eduardkoller

Yes, I think so, but in addition to that, we want to fix it on our side as well. Please help review and merge: https://github.com/Azure/azure-xplat-cli/pull/3250

Oct 12 '16 22:10 huangpf

It takes both true and "true". I haven't validated other cases, but I guess it will ignore cases for true false strings as well.

Oct 12 '16 23:10 huangpf

Ultimately, this has to be addressed on the server side. If the service wants to be permissive (i.e. support both strings and Booleans even though the documentation/contract is that it is supposed to be Booleans) then it can be so for data it receives. The service has to return data in the documented format/according to the contract. If not, there are no guarantees that any SDK behaves correctly - particularly for strongly typed languages.

Oct 13 '16 01:10 johanste

Is it the reopen reason? I'm fine with your position, but please make sure it's communicated to service team.

But just my 2 cents, a boolean defined type accepting a small set of possible values, i.e. true "true", or even 'True' seems to be user-friendly and not too difficult to imlement. BTW, are all swagger-spec generated languages strongly typed?

Oct 13 '16 01:10 huangpf

Not all SDKs are for strongly typed languages, but they most certainly do exist. C# and Java are two examples.

The swagger document also is the source for documentation for the API. And you should be able to use a code generator or runtime other than AutoRest (two examples I personally have happened to use are swagger-codegen and swagger-node, but there are most certainly others out there) in order to consume the service. Thus, making code changes in our SDKs (or in the CLI) to handle the incorrect behavior is not solving the broader problem.

Oct 13 '16 01:10 johanste

The scope of this problem doesn't have to be expanded to strongly typed languages. Actually various languages provide friendly boolean string parser functions, and in my opinion this shall be the scope of this problem in that whether you want to utilize them.

C#

Boolean.TryParse Method (String, Boolean)
//       'True' --> True
//       'False' --> False
//       'true' --> True
//       'false' --> False
//       '    true    ' --> True

Node.js

// You use this to convert 'true' to true in your get() function. Isn't it not strongly typed?
JSON.parse('true');            // true

Python

distutils.util.strtobool(val)
Convert a string representation of truth to true (1) or false (0).

True values are y, yes, t, true, on and 1; false values are n, no, f, false, off and 0. Raises ValueError if val is anything else.

Oct 13 '16 06:10 huangpf

@huangpf - We are not talking about the implementation details. We are talking about accuracy. The swagger spec should be an accurate representation of the service. The service should be strict in sending the response, it can be more open when accepting values. The suggestions that you are pointing to (the C# one) can also be implemented in the service, so that the service is consistent in sending it's response.

Btw, in node.js, JSON.parse is capable of parsing, number, boolean, string (basic data types); provided they are represented correctly. Input to JSON.parse is always a string. The server is sending a string true and not a boolean true. The example you showed is parsing a boolean true. Whereas the actual situation is this:

JSON.parse('"true"')' -> 'true' JSON.parse('" true "') -> ' true '

Oct 13 '16 15:10 amarzavery

@huangpf - Has this issue been fixed on the service side?

Jan 04 '17 00:01 amarzavery