security icon indicating copy to clipboard operation
security copied to clipboard
verified publisher icon opensearch-project

[Fix #4280] Introduce new endpoint _plugins/_security/api/certificates

Open willyborankin opened this issue 1 year ago • 2 comments

Description

Introduce new endpoint:

  • _plugins/_security/api/certificates
  • _plugins/_security/api/certificates/{nodeId}

which provides information about SSL certificates for each node in the cluster. The endpoint works only with Default key store but not with external key store.

Path parameters:

  • nodeId - (Optional, string) The names of particular nodes in the cluster to target. For example, nodeId1,nodeId2

Query string parameters:

  • cert_type - (Optional, string) The SSL certificate type. Expected values are: http, transport and all. The default value is all
  • timeout - node request timeout

HTTP Response:

{
  "_nodes" : {
    "total" : 0,
    "successful" : 0,
    "failed" : 0
  },
  "cluster_name" : "...",
  "nodes" : {
    "..." : {
      "name" : "...",
      "certificates" : {
        "http" : [{
          "issuer_dn" : "...",
          "not_after" : "...",
          "san" : "...",
          "subject_dn" : "...",
          "not_before" : "..."
        }, ...],
        "transport" : [{
          "issuer_dn" : "...",
          "not_after" : "...",
          "san" : "...",
          "subject_dn" : "...",
          "not_before" : "..."
        }, ...]
      }
    }
  }
}

Example HTTP response:

{
  "_nodes" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "cluster_name" : "local_cluster_1",
  "nodes" : {
    "fxxUhP__T_-hAn-A_____w" : {
      "name" : "cluster_manager_0",
      "certificates" : {
        "http" : [ {
          "issuer_dn" : "DC=com,DC=example,O=Example Com Inc.,OU=Example Com Inc. Root CA,CN=Example Com Inc. Root CA",
          "not_after" : "2025-04-28T09:23:47Z",
          "san" : "[[8, 1.2.3.4.5.5], [2, node-0.example.com], [2, localhost], [7, 127.0.0.1]]",
          "subject_dn" : "DC=de,L=test,O=node,OU=node,CN=node-0.example.com",
          "not_before" : "2024-04-28T09:23:47Z"
        } ],
        "transport" : [ {
          "issuer_dn" : "DC=com,DC=example,O=Example Com Inc.,OU=Example Com Inc. Root CA,CN=Example Com Inc. Root CA",
          "not_after" : "2025-04-28T09:23:47Z",
          "san" : "[[8, 1.2.3.4.5.5], [2, node-0.example.com], [2, localhost], [7, 127.0.0.1]]",
          "subject_dn" : "DC=de,L=test,O=node,OU=node,CN=node-0.example.com",
          "not_before" : "2024-04-28T09:23:47Z"
        } ]
      }
    },
    "i7LgEwAAQACap1QkAAAAAA" : {
      "name" : "cluster_manager_1",
      "certificates" : {
        "http" : [ {
          "issuer_dn" : "DC=com,DC=example,O=Example Com Inc.,OU=Example Com Inc. Root CA,CN=Example Com Inc. Root CA",
          "not_after" : "2025-04-28T09:23:47Z",
          "san" : "[[8, 1.2.3.4.5.5], [2, node-1.example.com], [2, localhost], [7, 127.0.0.1]]",
          "subject_dn" : "DC=de,L=test,O=node,OU=node,CN=node-1.example.com",
          "not_before" : "2024-04-28T09:23:47Z"
        } ],
        "transport" : [ {
          "issuer_dn" : "DC=com,DC=example,O=Example Com Inc.,OU=Example Com Inc. Root CA,CN=Example Com Inc. Root CA",
          "not_after" : "2025-04-28T09:23:47Z",
          "san" : "[[8, 1.2.3.4.5.5], [2, node-1.example.com], [2, localhost], [7, 127.0.0.1]]",
          "subject_dn" : "DC=de,L=test,O=node,OU=node,CN=node-1.example.com",
          "not_before" : "2024-04-28T09:23:47Z"
        } ]
      }
    },
    "58zZ5v__T_-GFzPu_____w" : {
      "name" : "cluster_manager_2",
      "certificates" : {
        "http" : [ {
          "issuer_dn" : "DC=com,DC=example,O=Example Com Inc.,OU=Example Com Inc. Root CA,CN=Example Com Inc. Root CA",
          "not_after" : "2025-04-28T09:23:47Z",
          "san" : "[[8, 1.2.3.4.5.5], [2, node-2.example.com], [2, localhost], [7, 127.0.0.1]]",
          "subject_dn" : "DC=de,L=test,O=node,OU=node,CN=node-2.example.com",
          "not_before" : "2024-04-28T09:23:47Z"
        } ],
        "transport" : [ {
          "issuer_dn" : "DC=com,DC=example,O=Example Com Inc.,OU=Example Com Inc. Root CA,CN=Example Com Inc. Root CA",
          "not_after" : "2025-04-28T09:23:47Z",
          "san" : "[[8, 1.2.3.4.5.5], [2, node-2.example.com], [2, localhost], [7, 127.0.0.1]]",
          "subject_dn" : "DC=de,L=test,O=node,OU=node,CN=node-2.example.com",
          "not_before" : "2024-04-28T09:23:47Z"
        } ]
      }
    },
    "YOYidQAAQACP5uIKAAAAAA" : {
      "name" : "data_0",
      "certificates" : {
        "http" : [ {
          "subject_dn" : "DC=de,L=test,O=node,OU=node,CN=node-3.example.com",
          "san" : "[[8, 1.2.3.4.5.5], [2, node-3.example.com], [2, localhost], [7, 127.0.0.1]]",
          "issuer_dn" : "DC=com,DC=example,O=Example Com Inc.,OU=Example Com Inc. Root CA,CN=Example Com Inc. Root CA",
          "not_after" : "2025-04-28T09:23:47Z",
          "not_before" : "2024-04-28T09:23:47Z"
        } ],
        "transport" : [ {
          "subject_dn" : "DC=de,L=test,O=node,OU=node,CN=node-3.example.com",
          "san" : "[[8, 1.2.3.4.5.5], [2, node-3.example.com], [2, localhost], [7, 127.0.0.1]]",
          "issuer_dn" : "DC=com,DC=example,O=Example Com Inc.,OU=Example Com Inc. Root CA,CN=Example Com Inc. Root CA",
          "not_after" : "2025-04-28T09:23:47Z",
          "not_before" : "2024-04-28T09:23:47Z"
        } ]
      }
    },
    "fcavYQAAQACJuOUMAAAAAA" : {
      "name" : "data_1",
      "certificates" : {
        "http" : [ {
          "issuer_dn" : "DC=com,DC=example,O=Example Com Inc.,OU=Example Com Inc. Root CA,CN=Example Com Inc. Root CA",
          "not_after" : "2025-04-28T09:23:47Z",
          "san" : "[[8, 1.2.3.4.5.5], [2, node-4.example.com], [2, localhost], [7, 127.0.0.1]]",
          "subject_dn" : "DC=de,L=test,O=node,OU=node,CN=node-4.example.com",
          "not_before" : "2024-04-28T09:23:47Z"
        } ],
        "transport" : [ {
          "issuer_dn" : "DC=com,DC=example,O=Example Com Inc.,OU=Example Com Inc. Root CA,CN=Example Com Inc. Root CA",
          "not_after" : "2025-04-28T09:23:47Z",
          "san" : "[[8, 1.2.3.4.5.5], [2, node-4.example.com], [2, localhost], [7, 127.0.0.1]]",
          "subject_dn" : "DC=de,L=test,O=node,OU=node,CN=node-4.example.com",
          "not_before" : "2024-04-28T09:23:47Z"
        } ]
      }
    }
  }
}

Issues Resolved

#4280

Testing

[Please provide details of testing done: unit testing, integration testing and manual testing]

Check List

  • [ ] New functionality includes testing
  • [ ] New functionality has been documented
  • [ ] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

willyborankin avatar Apr 29 '24 13:04 willyborankin

Took an initial pass. This approach looks great. Left some comments. I do see changes from #4252, was that accidental?

~I cherry-picked it since it contains fix for certificates generation for nodes.~

willyborankin avatar Apr 30 '24 22:04 willyborankin

Codecov Report

Attention: Patch coverage is 10.66667% with 201 lines in your changes are missing coverage. Please review.

Project coverage is 65.45%. Comparing base (d19a8ba) to head (9ab5e0c).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #4299      +/-   ##
==========================================
- Coverage   66.02%   65.45%   -0.57%     
==========================================
  Files         302      310       +8     
  Lines       21762    21986     +224     
  Branches     3523     3552      +29     
==========================================
+ Hits        14368    14391      +23     
- Misses       5626     5825     +199     
- Partials     1768     1770       +2
Files Coverage Δ
...security/dlic/rest/api/SecurityRestApiActions.java 80.00% <ø> (ø)
...urity/dlic/rest/api/SecuritySSLCertsApiAction.java 72.46% <100.00%> (+0.40%) :arrow_up:
...rity/dlic/rest/api/ssl/CertificatesActionType.java 100.00% <100.00%> (ø)
.../opensearch/security/OpenSearchSecurityPlugin.java 84.30% <50.00%> (-0.20%) :arrow_down:
...h/security/dlic/rest/api/ssl/CertificatesInfo.java 0.00% <0.00%> (ø)
...lic/rest/api/ssl/CertificatesInfoNodesRequest.java 0.00% <0.00%> (ø)
...ch/security/dlic/rest/api/ssl/CertificateType.java 0.00% <0.00%> (ø)
.../security/dlic/rest/api/CertificatesApiAction.java 36.11% <36.11%> (ø)
.../api/ssl/TransportCertificatesInfoNodesAction.java 10.52% <10.52%> (ø)
...ch/security/dlic/rest/api/ssl/CertificateInfo.java 0.00% <0.00%> (ø)
... and 1 more

codecov[bot] avatar May 01 '24 11:05 codecov[bot]

@willyborankin Great work! A couple of questions- what would the behavior be when-

  1. A cluster with security disabled is performing a rolling restart to enable security with transport layer certificates. Will this will generate an IllegalStateException, can you confirm?
  2. A mixed cluster where a previous version of OS doesn't understand the transport action.

shikharj05 avatar May 09 '24 11:05 shikharj05

@willyborankin Great work! A couple of questions- what would the behavior be when-

1. A cluster with security disabled is performing a rolling restart to enable security with transport layer certificates. Will this will generate an [`IllegalStateException`](https://github.com/opensearch-project/security/blob/2c07bfbefa513d8ace1de789416cf2fcbd69fad4/src/main/java/org/opensearch/security/dlic/rest/api/ssl/TransportCertificatesInfoNodesAction.java#L95), can you confirm?
  • if you have a dedicated coordinator node when you will be getting HTTP not implemented until the node starts to use the security plugin. As soon as the coordinator node started to use the security plugin you will get a failure response if other nodes still rebooting:

An example of failure response:


{
 "_nodes": {
   "total": 5,
   "successful": 1,
   "failed": 4,
   "failures": [
     {
       "type": "failed_node_exception",
       "reason": "Failed node [meWIrv__T_-3_hR0AAAAAA]",
       "node_id": "meWIrv__T_-3_hR0AAAAAA",
       "caused_by": {
         "type": "illegal_argument_exception",
         "reason": "some reason"
       }
     },
     {
       "type": "failed_node_exception",
       "reason": "Failed node [ierwfAAAQAC_luT5_____w]",
       "node_id": "ierwfAAAQAC_luT5_____w",
       "caused_by": {
         "type": "illegal_argument_exception",
         "reason": "some reason"
       }
     },
     {
       "type": "failed_node_exception",
       "reason": "Failed node [tRcffAAAQACcRK-n_____w]",
       "node_id": "tRcffAAAQACcRK-n_____w",
       "caused_by": {
         "type": "illegal_argument_exception",
         "reason": "some reason"
       }
     },
     {
       "type": "failed_node_exception",
       "reason": "Failed node [sWODKwAAQACeoJxjAAAAAA]",
       "node_id": "sWODKwAAQACeoJxjAAAAAA",
       "caused_by": {
         "type": "illegal_argument_exception",
         "reason": "some reason"
       }
     }
   ]
 },
 "cluster_name": "local_cluster_1",
 "nodes": {
   "YOYidQAAQACP5uIKAAAAAA" : {
     "name" : "data_0",
     "certificates" : {
       "http" : [ {
         "subject_dn" : "DC=de,L=test,O=node,OU=node,CN=node-3.example.com",
         "san" : "[[8, 1.2.3.4.5.5], [2, node-3.example.com], [2, localhost], [7, 127.0.0.1]]",
         "issuer_dn" : "DC=com,DC=example,O=Example Com Inc.,OU=Example Com Inc. Root CA,CN=Example Com Inc. Root CA",
         "not_after" : "2025-04-28T09:23:47Z",
         "not_before" : "2024-04-28T09:23:47Z"
       } ],
       "transport" : [ {
         "subject_dn" : "DC=de,L=test,O=node,OU=node,CN=node-3.example.com",
         "san" : "[[8, 1.2.3.4.5.5], [2, node-3.example.com], [2, localhost], [7, 127.0.0.1]]",
         "issuer_dn" : "DC=com,DC=example,O=Example Com Inc.,OU=Example Com Inc. Root CA,CN=Example Com Inc. Root CA",
         "not_after" : "2025-04-28T09:23:47Z",
         "not_before" : "2024-04-28T09:23:47Z"
       } ]
     }
   }  
}
}
  • if you do not have a dedicated coordinator node. It will fail randomly with not implemented and the response above.
2. A mixed cluster where a previous version of OS doesn't understand the transport action.

Same as above.

willyborankin avatar May 14 '24 19:05 willyborankin

@shikharj05 Does @willyborankin's comment address your questions? Since we have 2 approvals we will be merging soon if there are no more questions.

DarshitChanpura avatar May 15 '24 17:05 DarshitChanpura

@shikharj05 Does @willyborankin's comment address your questions? Since we have 2 approvals we will be merging soon if there are no more questions.

Thanks for the response @willyborankin. @DarshitChanpura - we should be good to merge, as we can see from the results above, the API doesn't fail in a mixed cluster.

shikharj05 avatar May 17 '24 17:05 shikharj05

@willyborankin Can you check the windows integration test CI failures once. They seem to be failing even after 3 retries. https://github.com/opensearch-project/security/actions/runs/9099275845/job/25114549425?pr=4299#step:4:31257

DarshitChanpura avatar May 17 '24 19:05 DarshitChanpura

@willyborankin Can you check the windows integration test CI failures once. They seem to be failing even after 3 retries. https://github.com/opensearch-project/security/actions/runs/9099275845/job/25114549425?pr=4299#step:4:31257

~~Yes it started to fail. Interesting :-)~~ Fixed.

willyborankin avatar May 18 '24 18:05 willyborankin

@willyborankin We should document this API on https://opensearch.org/docs/latest. Would you mind opening a doc PR against https://github.com/opensearch-project/documentation-website/?

DarshitChanpura avatar Jul 23 '24 05:07 DarshitChanpura