Mismatch values physical_used_wo_snapshots and physical_used_wo_snapshots_flexclones Zapi/Rest

[rahulguptajss]

What is the solution for value difference for aggr_physical_used_wo_snapshots, aggr_physical_used_wo_snapshots_flexclones

we are using this calculation in REST plugin:

        - total_physical_used SUBTRACT space.efficiency.logical_used space.efficiency.savings
        - physical_used_wo_snapshots SUBTRACT space.efficiency_without_snapshots.logical_used space.efficiency_without_snapshots.savings
        - physical_used_wo_snapshots_flexclones SUBTRACT space.efficiency_without_snapshots_flexclones.logical_used space.efficiency_without_snapshots_flexclones.savings

Note: It seems until we have efficiency_without_snapshots.savings value from REST response, the data mis match would exist.

To understand more, compare 9.12 cluster response as well as infinity response. In infinity out of 5 records, 4 values are matching and 1 is not matching where efficiency_without_snapshots.savings is 0.

Response from cluster 10.236.41.247: ZAPI

hardikl@hardikl-mac-0 harvest % ./bin/zapi --poller cluster-29 show data --api aggr-efficiency-get-iter | dasel -r xml -w json | grep total-data-reduction-physical-used-wo-snapshots              
connected to sti115nscluster-1 (NetApp Release Yellowdog__9.12.1: Mon Jun 27 06:27:46 UTC 2022)
          "total-data-reduction-physical-used-wo-snapshots": "1572864",
          "total-data-reduction-physical-used-wo-snapshots-flexclones": "1572864",

hardikl@hardikl-mac-0 harvest % ./bin/zapi --poller cluster-29 show data --api aggr-efficiency-get-iter | dasel -r xml -w json | grep total-data-reduction-logical-used-wo-snapshots
connected to sti115nscluster-1 (NetApp Release Yellowdog__9.12.1: Mon Jun 27 06:27:46 UTC 2022)
          "total-data-reduction-logical-used-wo-snapshots": "1069056",
          "total-data-reduction-logical-used-wo-snapshots-flexclones": "1069056",

REST

hardikl@hardikl-mac-0 harvest % ./bin/rest --poller cluster-29 show data --api storage/aggregates --fields "space.efficiency_without_snapshots.logical_used,space.efficiency_without_snapshots.savings"
fetching href=[api/storage/aggregates?return_records=true&fields=space.efficiency_without_snapshots.logical_used,space.efficiency_without_snapshots.savings]
{
 "records": [
  {
   "name": "sti115_vsim_ucs630h_aggr1",
   "space": {
    "efficiency_without_snapshots": {
     "logical_used": 1069056,
     "savings": 0
    }
   },
   "uuid": "35d9ba70-ee19-45a8-be94-2676440aa0c6"
  }
 ],
 "num_records": 1
}

Response from cluster infinity: ZAPI

hardikl@hardikl-mac-0 harvest % ./bin/zapi --poller cluster-21 show data --api aggr-efficiency-get-iter | dasel -r xml -w json | grep total-data-reduction-physical-used-wo-snapshots                
connected to ocum-infinity (NetApp Release Metropolitan__9.11.1: Fri Feb 04 05:24:36 UTC 2022)
            "total-data-reduction-physical-used-wo-snapshots": "1890805197595",
            "total-data-reduction-physical-used-wo-snapshots-flexclones": "1888353852700",
            "total-data-reduction-physical-used-wo-snapshots": "2242471822043",
            "total-data-reduction-physical-used-wo-snapshots-flexclones": "2242316889228",
            "total-data-reduction-physical-used-wo-snapshots": "1810289330818",
            "total-data-reduction-physical-used-wo-snapshots-flexclones": "1810289330818",
            "total-data-reduction-physical-used-wo-snapshots": "548092197088",
            "total-data-reduction-physical-used-wo-snapshots-flexclones": "548092197088",
            "total-data-reduction-physical-used-wo-snapshots": "304537600",
            "total-data-reduction-physical-used-wo-snapshots-flexclones": "304537600",

hardikl@hardikl-mac-0 harvest % ./bin/zapi --poller cluster-21 show data --api aggr-efficiency-get-iter | dasel -r xml -w json | grep total-data-reduction-logical-used-wo-snapshots
connected to ocum-infinity (NetApp Release Metropolitan__9.11.1: Fri Feb 04 05:24:36 UTC 2022)
            "total-data-reduction-logical-used-wo-snapshots": "6778318286848",
            "total-data-reduction-logical-used-wo-snapshots-flexclones": "6739645136896",
            "total-data-reduction-logical-used-wo-snapshots": "5505030373376",
            "total-data-reduction-logical-used-wo-snapshots-flexclones": "5493035769856",
            "total-data-reduction-logical-used-wo-snapshots": "4535282896896",
            "total-data-reduction-logical-used-wo-snapshots-flexclones": "4535282896896",
            "total-data-reduction-logical-used-wo-snapshots": "2261531185152",
            "total-data-reduction-logical-used-wo-snapshots-flexclones": "2261531185152",
            "total-data-reduction-logical-used-wo-snapshots": "618496",
            "total-data-reduction-logical-used-wo-snapshots-flexclones": "618496",

REST

hardikl@hardikl-mac-0 harvest % ./bin/rest --poller cluster-21 show data --api storage/aggregates --fields "space.efficiency_without_snapshots.logical_used,space.efficiency_without_snapshots.savings"
fetching href=[api/storage/aggregates?return_records=true&fields=space.efficiency_without_snapshots.logical_used,space.efficiency_without_snapshots.savings]
{
 "records": [
  {
   "name": "aggr_snap",
   "space": {
    "efficiency_without_snapshots": {
     "logical_used": 618496,
     "savings": 0
    }
   },
   "uuid": "1b222034-2274-4e1f-8e5f-95396067d5e8"
  },
  {
   "name": "aggr2",
   "space": {
    "efficiency_without_snapshots": {
     "logical_used": 5505030381568,
     "savings": 3262558412260
    }
   },
   "uuid": "8072dcd4-1118-4f3d-8564-4fff9dc80653"
  },
  {
   "name": "aggr4",
   "space": {
    "efficiency_without_snapshots": {
     "logical_used": 2261531185152,
     "savings": 1713438840159
    }
   },
   "uuid": "c8d8c2d6-bde6-4fdf-b152-6d595d30e4f8"
  },
  {
   "name": "aggr1",
   "space": {
    "efficiency_without_snapshots": {
     "logical_used": 6778318295040,
     "savings": 4887512989866
    }
   },
   "uuid": "e2ce6f5e-73d9-4994-b8ee-9e7c43a4080c"
  },
  {
   "name": "aggr3",
   "space": {
    "efficiency_without_snapshots": {
     "logical_used": 4535282896896,
     "savings": 2724993600208
    }
   },
   "uuid": "eb522c12-0124-4740-bae3-987523cade6d"
  }
 ],
 "num_records": 5
}

Originally posted by @Hardikl in https://github.com/NetApp/harvest/issues/1168#issuecomment-1195264356