.to_json() should change null to None rather than skip the KV pair

[scaleoutsean]

• SolidFire SDK Python v 12.3

In JSON-RPC responses some values may be null:

{
    "id": 1,
    "result": {
        "volumes": [
            {
                "access": "readWrite",
                "accountID": 1,
                "attributes": {},
                "blockSize": 4096,
                "createTime": "2023-02-27T17:04:09Z",
                "currentProtectionScheme": "singleHelix",
                "deleteTime": "",
                "enable512e": true,
                "enableSnapMirrorReplication": false,
                "fifoSize": 5,
                "iqn": "iqn.2010-01.com.solidfire:wcwb.data.1",
                "lastAccessTime": null,
                "lastAccessTimeIO": null,
                "minFifoSize": 0,
                "name": "data",
                "previousProtectionScheme": null,
                "purgeTime": "",
                "qos": {
                    "burstIOPS": 3000,
                    "burstTime": 60,
                    "curve": {
                        "1048576": 15000,
                        "131072": 1950,
                        "16384": 270,
                        "262144": 3900,
                        "32768": 500,
                        "4096": 100,
                        "524288": 7600,
                        "65536": 1000,
                        "8192": 160
                    },
                    "maxIOPS": 1500,
                    "minIOPS": 100
                },
                "qosPolicyID": 2,
                "scsiEUIDeviceID": "7763776200000001f47acc0100000000",
                "scsiNAADeviceID": "6f47acc1000000007763776200000001",
                "sliceCount": 1,
                "status": "active",
                "totalSize": 2000683008,
                "virtualVolumeID": null,
                "volumeAccessGroups": [],
                "volumeConsistencyGroupUUID": "e6b28e75-c21c-4bad-8a2e-056bcec14e4c",
                "volumeID": 1,
                "volumePairs": [],
                "volumeUUID": "4b31894c-b3d4-4dbc-a438-d60aa96f7c49"
            }
        ]
    }
}

Below we can see that possibility in qos_policy_id=None (correct).

>>> sfe.list_volumes()
...
Volume(access=VolumeAccess(), 
account_id=6, attributes={}, block_size=4096, create_time=\'2023-11-19T09:59:36Z\',
current_protection_scheme=ProtectionScheme(), delete_time=\'\', enable512e=True,
enable_snap_mirror_replication=False, fifo_size=5,
iqn=\'iqn.2010-01.com.solidfire:wcwb.dfasdfa.62\', last_access_time=None, last_access_time_io=None,
min_fifo_size=0, name=\'dfasdfa\', previous_protection_scheme=None, purge_time=\'\',
qos=VolumeQOS(burst_iops=15000, burst_time=60, 
curve={\'1048576\': 15000, \'131072\': 1950, \'16384\': 270, \'262144\': 3900, \'32768\': 500, \'4096\': 100, \'524288\': 7600, \'65536\': 1000, \'8192\': 160}, max_iops=15000, min_iops=50),
qos_policy_id=None, scsi_euidevice_id=\'776377620000003ef47acc0100000000\', 
scsi_naadevice_id=\'6f47acc100000000776377620000003e\', slice_count=1, status=\'active\', 
total_size=2000683008, virtual_volume_id=None, volume_access_groups=\'[]\', 
volume_consistency_group_uuid=UUID(\'862a51f0-2ca2-46b1-a6b2-1fafde67ff76\'), volume_id=62, 
volume_pairs=\'[]\', volume_uuid=UUID(\'ddefa4a4-6753-427b-9869-755b77d29814\'))]')

Instead of preserving that in JSON, list_volumes().to_json() completely drops those KVs. Then code that expects to find 0 or None trips on this rendering code very unpredictable.

>>> list_volumes[16]
{'access': {}, 'accountID': 6, 'attributes': {}, 'blockSize': 4096, 'createTime': '2023-11-19T09:59:36Z', 'currentProtectionScheme': {}, 'deleteTime': '', 'enable512e': True, 'enableSnapMirrorReplication': False, 'fifoSize': 5, 'iqn': 'iqn.2010-01.com.solidfire:wcwb.dfasdfa.62', 'minFifoSize': 0, 'name': 'dfasdfa', 'purgeTime': '', 'qos': {'burstIOPS': 15000, 'burstTime': 60, 'curve': {'1048576': 15000, '131072': 1950, '16384': 270, '262144': 3900, '32768': 500, '4096': 100, '524288': 7600, '65536': 1000, '8192': 160}, 'maxIOPS': 15000, 'minIOPS': 50}, 'scsiEUIDeviceID': '776377620000003ef47acc0100000000', 'scsiNAADeviceID': '6f47acc100000000776377620000003e', 'sliceCount': 1, 'status': 'active', 'totalSize': 2000683008, 'volumeAccessGroups': [], 'volumeConsistencyGroupUUID': '862a51f0-2ca2-46b1-a6b2-1fafde67ff76', 'volumeID': 62, 'volumePairs': [], 'volumeUUID': 'ddefa4a4-6753-427b-9869-755b77d29814'}
>>> 
>>> list_volumes[1]
{'access': {}, 'accountID': 1, 'attributes': {}, 'blockSize': 4096, 'createTime': '2023-02-27T17:04:20Z', 'currentProtectionScheme': {}, 'deleteTime': '', 'enable512e': True, 'enableSnapMirrorReplication': False, 'fifoSize': 5, 'iqn': 'iqn.2010-01.com.solidfire:wcwb.log.2', 'minFifoSize': 0, 'name': 'log', 'purgeTime': '', 'qos': {'burstIOPS': 1000, 'burstTime': 60, 'curve': {'1048576': 15000, '131072': 1950, '16384': 270, '262144': 3900, '32768': 500, '4096': 100, '524288': 7600, '65536': 1000, '8192': 160}, 'maxIOPS': 800, 'minIOPS': 100}, 'qosPolicyID': 1, 'scsiEUIDeviceID': '7763776200000002f47acc0100000000', 'scsiNAADeviceID': '6f47acc1000000007763776200000002', 'sliceCount': 1, 'status': 'active', 'totalSize': 1073741824, 'volumeAccessGroups': [], 'volumeConsistencyGroupUUID': '6dc2588d-bedf-4a64-a958-e8377c38f551', 'volumeID': 2, 'volumePairs': [], 'volumeUUID': '8721d0a3-735a-4cad-9502-15b09827f710'}
>>> 
>>> list_volumes[1]['qosPolicyID']
1
>>> 
>>> list_volumes[15]['qosPolicyID']
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyError: 'qosPolicyID'

Volume with a QoS Policy ID 1 can work today, but if the user switches to Custom for whatever reason, now the key is completely missing, and only for that volume.

[scaleoutsean]

Of course, since sfe.list_volumes() and other methods work correctly prior to .to_json(), a workaround is to convert to JSON by yourself.