Scrape ONTAP S3 Metrics

[Sawall10]

Is your feature request related to a problem? Please describe. Scrap and report on ONTAP S3 metrics

Describe the solution you'd like Report on any and all ONTAP S3 metrics. Similar concept to how NFS is today

Describe alternatives you've considered None

Additional context None

[cgrinds]

Thanks @Sawall10 looks like there are APIs for everything. We'll add some templates for the APIs under object-store.

Example of what's avaliable (if you want to try locally, replace $POLLER with your poller name)

bin/rest --poller $POLLER show data --api protocols/s3/buckets

{
 "records": [
  {
   "comment": "",
   "encryption": {
    "enabled": false
   },
   "logical_used_size": 0,
   "name": "bucket1",
   "policy": {
    "statements": [
     {
      "actions": [
       "GetObject",
       "PutObject",
       "DeleteObject",
       "ListBucket",
       "GetBucketAcl",
       "GetObjectAcl",
       "ListBucketMultipartUploads",
       "ListMultipartUploadParts",
       "GetObjectTagging",
       "PutObjectTagging",
       "DeleteObjectTagging",
       "GetBucketLocation",
       "GetBucketVersioning",
       "PutBucketVersioning",
       "ListBucketVersions"
      ],
      "effect": "allow",
      "principals": [],
      "resources": [
       "bucket1",
       "bucket1/*"
      ],
      "sid": ""
     }
    ]
   },
   "protection_status": {
    "destination": {
     "is_cloud": false,
     "is_ontap": false
    },
    "is_protected": false
   },
   "role": "standalone",
   "size": 16492674416640,
   "svm": {
    "name": "svm_k8",
    "uuid": "uid1"
   },
   "uuid": "uid2",
   "versioning_state": "disabled",
   "volume": {
    "name": "fg_oss_1661971987",
    "uuid": "uid5"
   }
  },
  {
   "comment": "",
   "encryption": {
    "enabled": false
   },
   "logical_used_size": 1638342656,
   "name": "bucket2",
   "protection_status": {
    "destination": {
     "is_cloud": false,
     "is_ontap": false
    },
    "is_protected": false
   },
   "role": "standalone",
   "size": 21990232555520,
   "svm": {
    "name": "svm_k8",
    "uuid": "uid1"
   },
   "uuid": "uid6",
   "versioning_state": "disabled",
   "volume": {
    "name": "fg_oss_1661982724",
    "uuid": "uid7"
   }
  }
 ],
 "num_records": 2
}

[cgrinds]

Few ZAPI related notes - we haven't figured out a config ZAPI that collects S3 buckets yet. These are perf ZAPIs objects.

    <name>object_store_server</name>
    <name>object_store_server:node</name>
    <name>object_store_server:vserver</name>

bin/zapi --poller cpoc show counters --object object_store_server | rg -F "[name]" | rg latency | rg -v 'histogram|base'

connected to a800-1 (NetApp Release 9.11.1P1: Tue Aug 09 13:13:19 UTC 2022)
    [name]                                         -      abort_multipart_upload_latency
    [name]                                         -   complete_multipart_upload_latency
    [name]                                         -               create_bucket_latency
    [name]                                         -               delete_bucket_latency
    [name]                                         -               delete_object_latency
    [name]                                         -       delete_object_tagging_latency
    [name]                                         -         get_object_lastbyte_latency
    [name]                                         -                  get_object_latency
    [name]                                         -          get_object_tagging_latency
    [name]                                         -                 head_bucket_latency
    [name]                                         -                 head_object_latency
    [name]                                         -   initiate_multipart_upload_latency
    [name]                                         -                list_buckets_latency
    [name]                                         -        list_object_versions_latency
    [name]                                         -                list_objects_latency
    [name]                                         -                list_uploads_latency
    [name]                                         -                  put_object_latency
    [name]                                         -          put_object_tagging_latency
    [name]                                         -                 upload_part_latency

bin/zapi --poller cpoc show data --object object_store_server --counter "instance_uuid,vserver_name,get_bucket_acl_total,head_bucket_total" | dasel -r xml -w json | pbcopy

{
  "root": {
    "instances": {
      "instance-data": [
        {
          "counters": {
            "counter-data": [
              {
                "name": "get_bucket_acl_total",
                "value": "0"
              },
              {
                "name": "head_bucket_total",
                "value": "145"
              },
              {
                "name": "instance_uuid",
                "value": "a800-1-01:kernel:s3.cpoc.local"
              },
              {
                "name": "vserver_name",
                "value": "svm_k8"
              }
            ]
          },
          "name": "s3.cpoc.local",
          "sort-id": "0",
          "uuid": "a800-1-01:kernel:s3.cpoc.local"
        },
        {
          "counters": {
            "counter-data": [
              {
                "name": "get_bucket_acl_total",
                "value": "0"
              },
              {
                "name": "head_bucket_total",
                "value": "32"
              },
              {
                "name": "instance_uuid",
                "value": "a800-1-02:kernel:s3.cpoc.local"
              },
              {
                "name": "vserver_name",
                "value": "svm_k8"
              }
            ]
          },
          "name": "s3.cpoc.local",
          "sort-id": "0",
          "uuid": "a800-1-02:kernel:s3.cpoc.local"
        }
      ]
    },
    "timestamp": "1663182324"
  }
}

[roock]

I think this was introduced in 22.11.0 and can be closed.