content icon indicating copy to clipboard operation
content copied to clipboard

CMP 2196 update ingress operator ciphers

Open rhmdnd opened this issue 1 year ago • 32 comments
trafficstars

The following ciphers are all supported with TLS v1.3, but we weren't checking for them in the OpenShift ingress controller configuration:

  • TLS_AES_128_GCM_SHA256
  • TLS_AES_256_GCM_SHA384
  • TLS_CHACHA20_POLY1305_SHA256

This commit updates the regular expression in the rule to check for those ciphers so the check doesn't fail if OpenShift is using them.

It also add some formatting to the rule so it's consistent with other TLS-related rules, like for the API server.

The following ciphers were listed in the "old" profile, or insecure, which should only be used as a last resort for server TLS configuration:

  • AES128-GCM-SHA256
  • AES256-GCM-SHA384
  • DHE-RSA-AES128-GCM-SHA256
  • DHE-RSA-AES256-GCM-SHA384

This commit removes them from the ingress controller rule so that it fails if a cluster is using these ciphers.

References:

  • https://wiki.mozilla.org/Security/Server_Side_TLS
  • https://docs.openssl.org/1.1.1/man1/ciphers/

Hints for reviewers - you'll need to make a tailored profile to test this rule since it's not included in any profiles by default.

---
apiVersion: compliance.openshift.io/v1alpha1
kind: TailoredProfile
metadata:
  name: upstream-scratch
spec:
  description: Modified CIS profile
  title: My modified CIS profile
  enableRules:
    - name: upstream-ocp4-kubelet-configure-tls-cipher-suites-ingresscontroller
      rationale: Included for testing
---
apiVersion: compliance.openshift.io/v1alpha1
kind: ScanSettingBinding
metadata:
  name: scratch-upstream-binding
  namespace: openshift-compliance
profiles:
  - name: upstream-scratch
    kind: TailoredProfile
    apiGroup: compliance.openshift.io/v1alpha1
settingsRef:
  name: default
  kind: ScanSetting
  apiGroup: compliance.openshift.io/v1alpha1

rhmdnd avatar Jul 24 '24 21:07 rhmdnd

Start a new ephemeral environment with changes proposed in this pull request:

ocp4 (from CTF) Environment (using Fedora as testing environment) Open in Gitpod

Fedora Testing Environment Open in Gitpod

Oracle Linux 8 Environment Open in Gitpod

github-actions[bot] avatar Jul 24 '24 21:07 github-actions[bot]

:robot: A k8s content image for this PR is available at: ghcr.io/complianceascode/k8scontent:12220 This image was built from commit: 3c24d289a2eb4df9476542197623b65c72d40232

Click here to see how to deploy it

If you alread have Compliance Operator deployed: utils/build_ds_container.py -i ghcr.io/complianceascode/k8scontent:12220

Otherwise deploy the content and operator together by checking out ComplianceAsCode/compliance-operator and: CONTENT_IMAGE=ghcr.io/complianceascode/k8scontent:12220 make deploy-local

github-actions[bot] avatar Jul 24 '24 21:07 github-actions[bot]

I think the OCIL has to be changed too. While the patch command would make the system compliant, there would be no ciphers for TLS1.3, which (might?) put the system in a state where it cant speak TLS1.3 because no ciphers are available?

Yes - good catch. I can update.

rhmdnd avatar Jul 25 '24 12:07 rhmdnd

This datastream diff is auto generated by the check Compare DS/Generate Diff

Click here to see the full diff
New content has different text for rule 'xccdf_org.ssgproject.content_rule_kubelet_configure_tls_cipher_suites_ingresscontroller'.
--- xccdf_org.ssgproject.content_rule_kubelet_configure_tls_cipher_suites_ingresscontroller
+++ xccdf_org.ssgproject.content_rule_kubelet_configure_tls_cipher_suites_ingresscontroller
@@ -10,7 +10,7 @@
 Therefore, you need to use a tool that can query the OCP API, retrieve the /apis/operator.openshift.io/v1/namespaces/openshift-ingress-operator/ingresscontrollers/default API endpoint to the local /apis/operator.openshift.io/v1/namespaces/openshift-ingress-operator/ingresscontrollers/default file.
 
 [reference]:
-4.2.13
+4.2.12
 
 [rationale]:
 TLS ciphers have had a number of known vulnerabilities and weaknesses,

OCIL for rule 'xccdf_org.ssgproject.content_rule_kubelet_configure_tls_cipher_suites_ingresscontroller' differs.
--- ocil:ssg-kubelet_configure_tls_cipher_suites_ingresscontroller_ocil:questionnaire:1
+++ ocil:ssg-kubelet_configure_tls_cipher_suites_ingresscontroller_ocil:questionnaire:1
@@ -1,4 +1,4 @@
 Run the following command on the kubelet nodes(s):
-oc -n openshift-ingress-operator patch ingresscontroller/default --type merge -p '{"spec":{"tlsSecurityProfile":{"type":"Custom","custom":{"ciphers":["ECDHE-ECDSA-AES128-GCM-SHA256","ECDHE-RSA-AES128-GCM-SHA256","ECDHE-RSA-AES256-GCM-SHA384"],"minTLSVersion":"VersionTLS12"} } } }'
-      Is it the case that TLS cipher suite configuration is not configured?
+oc -n openshift-ingress-operator patch ingresscontroller/default --type merge -p '{"spec":{"tlsSecurityProfile":{"type":"Custom","custom":{"ciphers":["ECDHE-ECDSA-AES128-GCM-SHA256","ECDHE-ECDSA-CHACHA20-POLY1305","ECDHE-ECDSA-AES256-GCM-SHA384","TLS_CHACHA20_POLY1305_SHA256","TLS_AES_128_GCM_SHA256","TLS_AES_256_GCM_SHA384","ECDHE-RSA-AES128-GCM-SHA256","ECDHE-RSA-AES256-GCM-SHA384","ECDHE-RSA-CHACHA20-POLY1305"],"minTLSVersion":"VersionTLS12"} } } }'
+      Is it the case that Ingress controller TLS cipher suite configuration is incomplete or possibly insecure?
       
kubernetes remediation for rule 'xccdf_org.ssgproject.content_rule_kubelet_configure_tls_cipher_suites_ingresscontroller' differs.
--- xccdf_org.ssgproject.content_rule_kubelet_configure_tls_cipher_suites_ingresscontroller
+++ xccdf_org.ssgproject.content_rule_kubelet_configure_tls_cipher_suites_ingresscontroller
@@ -10,6 +10,12 @@
       ciphers:
       - ECDHE-ECDSA-AES128-GCM-SHA256
       - ECDHE-RSA-AES128-GCM-SHA256
+      - ECDHE-ECDSA-CHACHA20-POLY1305
       - ECDHE-RSA-AES256-GCM-SHA384
+      - ECDHE-RSA-CHACHA20-POLY1305
+      - ECDHE-ECDSA-AES256-GCM-SHA384
+      - TLS_AES_128_GCM_SHA256
+      - TLS_AES_256_GCM_SHA384
+      - TLS_CHACHA20_POLY1305_SHA256
       minTLSVersion: VersionTLS12
     type: Custom

github-actions[bot] avatar Jul 26 '24 18:07 github-actions[bot]

/hold for test

xiaojiey avatar Aug 07 '24 08:08 xiaojiey

@rhmdnd According to https://access.redhat.com/articles/5348961, TLS_AES_128_CCM_SHA256 is also a new cipher suite new in TLS 1.3. Could you please double check we need to add it or not? Thanks.

xiaojiey avatar Aug 07 '24 08:08 xiaojiey

@rhmdnd Another issue is: seems the autoremediation cr will be in error status and will always fail due to permission issue: Unable to get fix object from ComplianceRemediation. Please update the compliance-operator's permissions: ingresscontrollers.operator.openshift.io \"default\" is forbidden: User \"system:serviceaccount:openshift-compliance:compliance-operator\" cannot get resource \"ingresscontrollers\" in API group \"operator.openshift.io\" in the namespace \"openshift-ingress-operator\"

Details seen from below:

% cat ssb_cis_p.yaml 
apiVersion: compliance.openshift.io/v1alpha1
kind: ScanSettingBinding
metadata:
  name: cis-compliance
  namespace: openshift-compliance
profiles:
  - name: upstream-ocp4-cis
    kind: Profile
    apiGroup: compliance.openshift.io/v1alpha1
settingsRef:
  name: default-auto-apply
  kind: ScanSetting
  apiGroup: compliance.openshift.io/v1alpha1
  % oc apply -f ssb_cis_p.yaml 
scansettingbinding.compliance.openshift.io/cis-compliance created
% oc get suite -w
NAME             PHASE     RESULT
cis-compliance   RUNNING   NOT-AVAILABLE
cis-compliance   AGGREGATING   NOT-AVAILABLE
cis-compliance   DONE          NON-COMPLIANT
cis-compliance   DONE          NON-COMPLIANT
^C%                                                                                                                                                                                                                 % oc get cr
NAME                                                                      STATE
upstream-ocp4-cis-kubelet-configure-tls-cipher-suites-ingresscontroller   Error
% oc get cr upstream-ocp4-cis-kubelet-configure-tls-cipher-suites-ingresscontroller -o=jsonpath={.status} | jq -r
{
  "applicationState": "Error",
  "errorMessage": "Unable to get fix object from ComplianceRemediation. Please update the compliance-operator's permissions: ingresscontrollers.operator.openshift.io \"default\" is forbidden: User \"system:serviceaccount:openshift-compliance:compliance-operator\" cannot get resource \"ingresscontrollers\" in API group \"operator.openshift.io\" in the namespace \"openshift-ingress-operator\""
}

xiaojiey avatar Aug 07 '24 12:08 xiaojiey

@rhmdnd Another issue is: seems the autoremediation cr will be in error status and will always fail due to permission issue: Unable to get fix object from ComplianceRemediation. Please update the compliance-operator's permissions: ingresscontrollers.operator.openshift.io \"default\" is forbidden: User \"system:serviceaccount:openshift-compliance:compliance-operator\" cannot get resource \"ingresscontrollers\" in API group \"operator.openshift.io\" in the namespace \"openshift-ingress-operator\"

Details seen from below:

% cat ssb_cis_p.yaml 
apiVersion: compliance.openshift.io/v1alpha1
kind: ScanSettingBinding
metadata:
  name: cis-compliance
  namespace: openshift-compliance
profiles:
  - name: upstream-ocp4-cis
    kind: Profile
    apiGroup: compliance.openshift.io/v1alpha1
settingsRef:
  name: default-auto-apply
  kind: ScanSetting
  apiGroup: compliance.openshift.io/v1alpha1
  % oc apply -f ssb_cis_p.yaml 
scansettingbinding.compliance.openshift.io/cis-compliance created
% oc get suite -w
NAME             PHASE     RESULT
cis-compliance   RUNNING   NOT-AVAILABLE
cis-compliance   AGGREGATING   NOT-AVAILABLE
cis-compliance   DONE          NON-COMPLIANT
cis-compliance   DONE          NON-COMPLIANT
^C%                                                                                                                                                                                                                 % oc get cr
NAME                                                                      STATE
upstream-ocp4-cis-kubelet-configure-tls-cipher-suites-ingresscontroller   Error
% oc get cr upstream-ocp4-cis-kubelet-configure-tls-cipher-suites-ingresscontroller -o=jsonpath={.status} | jq -r
{
  "applicationState": "Error",
  "errorMessage": "Unable to get fix object from ComplianceRemediation. Please update the compliance-operator's permissions: ingresscontrollers.operator.openshift.io \"default\" is forbidden: User \"system:serviceaccount:openshift-compliance:compliance-operator\" cannot get resource \"ingresscontrollers\" in API group \"operator.openshift.io\" in the namespace \"openshift-ingress-operator\""
}

Interesting - we proposed a separate permission fix to the operator and it landed recently.

https://github.com/ComplianceAsCode/compliance-operator/pull/558

Were you able to recreate with that fix?

rhmdnd avatar Aug 07 '24 12:08 rhmdnd

@rhmdnd Another issue is: seems the autoremediation cr will be in error status and will always fail due to permission issue: Unable to get fix object from ComplianceRemediation. Please update the compliance-operator's permissions: ingresscontrollers.operator.openshift.io \"default\" is forbidden: User \"system:serviceaccount:openshift-compliance:compliance-operator\" cannot get resource \"ingresscontrollers\" in API group \"operator.openshift.io\" in the namespace \"openshift-ingress-operator\" Details seen from below:

% cat ssb_cis_p.yaml 
apiVersion: compliance.openshift.io/v1alpha1
kind: ScanSettingBinding
metadata:
  name: cis-compliance
  namespace: openshift-compliance
profiles:
  - name: upstream-ocp4-cis
    kind: Profile
    apiGroup: compliance.openshift.io/v1alpha1
settingsRef:
  name: default-auto-apply
  kind: ScanSetting
  apiGroup: compliance.openshift.io/v1alpha1
  % oc apply -f ssb_cis_p.yaml 
scansettingbinding.compliance.openshift.io/cis-compliance created
% oc get suite -w
NAME             PHASE     RESULT
cis-compliance   RUNNING   NOT-AVAILABLE
cis-compliance   AGGREGATING   NOT-AVAILABLE
cis-compliance   DONE          NON-COMPLIANT
cis-compliance   DONE          NON-COMPLIANT
^C%                                                                                                                                                                                                                 % oc get cr
NAME                                                                      STATE
upstream-ocp4-cis-kubelet-configure-tls-cipher-suites-ingresscontroller   Error
% oc get cr upstream-ocp4-cis-kubelet-configure-tls-cipher-suites-ingresscontroller -o=jsonpath={.status} | jq -r
{
  "applicationState": "Error",
  "errorMessage": "Unable to get fix object from ComplianceRemediation. Please update the compliance-operator's permissions: ingresscontrollers.operator.openshift.io \"default\" is forbidden: User \"system:serviceaccount:openshift-compliance:compliance-operator\" cannot get resource \"ingresscontrollers\" in API group \"operator.openshift.io\" in the namespace \"openshift-ingress-operator\""
}

Interesting - we proposed a separate permission fix to the operator and it landed recently.

ComplianceAsCode/compliance-operator#558

Were you able to recreate with that fix?

No. Sorry. Let me use master branch to verify it again.

xiaojiey avatar Aug 08 '24 00:08 xiaojiey

@rhmdnd I tried catalogsource ghcr.io/complianceascode/compliance-operator-catalog:558 and ghcr.io/complianceascode/compliance-operator-catalog:573, I still got the same error. Could you please help to double check? Thanks.

xiaojiey avatar Aug 08 '24 04:08 xiaojiey

@rhmdnd I tried catalogsource ghcr.io/complianceascode/compliance-operator-catalog:558 and ghcr.io/complianceascode/compliance-operator-catalog:573, I still got the same error. Could you please help to double check? Thanks.

Strange - I wonder if the catalog source it's updating.

I tried with commit ff6c7de96c1a7f38ac37243640703858d035b7cc on https://github.com/ComplianceAsCode/compliance-operator using make deploy-local and that seems to have deployed the permission change.

I was able to use the following binding:

apiVersion: compliance.openshift.io/v1alpha1
kind: ScanSettingBinding
metadata:
  name: upstream-cis
  namespace: openshift-compliance
profiles:
  - name: upstream-ocp4-cis
    kind: Profile
    apiGroup: compliance.openshift.io/v1alpha1
  - name: upstream-ocp4-cis-node
    kind: Profile
    apiGroup: compliance.openshift.io/v1alpha1
settingsRef:
  name: default
  kind: ScanSetting
  apiGroup: compliance.openshift.io/v1alpha1

And then I applied the remediation through the Compliance Operator by setting apply: true. The upstream-ocp4-cis-kubelet-configure-tls-cipher-suites-ingresscontroller ComplianceCheckResult passed after rerunning the scan.

I was also able to verify this with the deploy technique mentioned in the comment above, using ff6c7de96c1a7f38ac37243640703858d035b7cc from ComplianceAsCode/compliance-operator.

$ CONTENT_IMAGE=ghcr.io/complianceascode/k8scontent:12220 make deploy-local

Do either of those techniques work for you?

rhmdnd avatar Aug 08 '24 17:08 rhmdnd

Sorry. Seems the PR image issue. I used make deploy-local and content image ghcr.io/complianceascode/k8scontent:12220. Didn't reproduce the issue.

xiaojiey avatar Aug 09 '24 06:08 xiaojiey

Then https://github.com/ComplianceAsCode/content/pull/12220#issuecomment-2272899368 this comment will be the only question left.

xiaojiey avatar Aug 09 '24 06:08 xiaojiey

Then #12220 (comment) this comment will be the only question left.

Good catch - fixed.

rhmdnd avatar Aug 09 '24 17:08 rhmdnd

/test

rhmdnd avatar Aug 09 '24 17:08 rhmdnd

@rhmdnd: The /test command needs one or more targets. The following commands are available to trigger required jobs:

  • /test 4.12-e2e-aws-ocp4-cis
  • /test 4.12-e2e-aws-ocp4-cis-node
  • /test 4.12-e2e-aws-ocp4-e8
  • /test 4.12-e2e-aws-ocp4-high
  • /test 4.12-e2e-aws-ocp4-high-node
  • /test 4.12-e2e-aws-ocp4-moderate
  • /test 4.12-e2e-aws-ocp4-moderate-node
  • /test 4.12-e2e-aws-ocp4-pci-dss
  • /test 4.12-e2e-aws-ocp4-pci-dss-node
  • /test 4.12-e2e-aws-ocp4-stig
  • /test 4.12-e2e-aws-ocp4-stig-node
  • /test 4.12-e2e-aws-rhcos4-e8
  • /test 4.12-e2e-aws-rhcos4-high
  • /test 4.12-e2e-aws-rhcos4-moderate
  • /test 4.12-e2e-aws-rhcos4-stig
  • /test 4.12-images
  • /test 4.13-e2e-aws-ocp4-bsi
  • /test 4.13-e2e-aws-ocp4-bsi-node
  • /test 4.13-e2e-aws-ocp4-cis
  • /test 4.13-e2e-aws-ocp4-cis-node
  • /test 4.13-e2e-aws-ocp4-e8
  • /test 4.13-e2e-aws-ocp4-high
  • /test 4.13-e2e-aws-ocp4-high-node
  • /test 4.13-e2e-aws-ocp4-moderate
  • /test 4.13-e2e-aws-ocp4-moderate-node
  • /test 4.13-e2e-aws-ocp4-pci-dss
  • /test 4.13-e2e-aws-ocp4-pci-dss-node
  • /test 4.13-e2e-aws-ocp4-stig
  • /test 4.13-e2e-aws-ocp4-stig-node
  • /test 4.13-e2e-aws-rhcos4-bsi
  • /test 4.13-e2e-aws-rhcos4-e8
  • /test 4.13-e2e-aws-rhcos4-high
  • /test 4.13-e2e-aws-rhcos4-moderate
  • /test 4.13-e2e-aws-rhcos4-stig
  • /test 4.13-images
  • /test 4.14-e2e-aws-ocp4-bsi
  • /test 4.14-e2e-aws-ocp4-bsi-node
  • /test 4.14-e2e-aws-rhcos4-bsi
  • /test 4.14-images
  • /test 4.15-e2e-aws-ocp4-bsi
  • /test 4.15-e2e-aws-ocp4-bsi-node
  • /test 4.15-e2e-aws-ocp4-cis
  • /test 4.15-e2e-aws-ocp4-cis-node
  • /test 4.15-e2e-aws-ocp4-e8
  • /test 4.15-e2e-aws-ocp4-high
  • /test 4.15-e2e-aws-ocp4-high-node
  • /test 4.15-e2e-aws-ocp4-moderate
  • /test 4.15-e2e-aws-ocp4-moderate-node
  • /test 4.15-e2e-aws-ocp4-pci-dss
  • /test 4.15-e2e-aws-ocp4-pci-dss-node
  • /test 4.15-e2e-aws-ocp4-stig
  • /test 4.15-e2e-aws-ocp4-stig-node
  • /test 4.15-e2e-aws-rhcos4-bsi
  • /test 4.15-e2e-aws-rhcos4-e8
  • /test 4.15-e2e-aws-rhcos4-high
  • /test 4.15-e2e-aws-rhcos4-moderate
  • /test 4.15-e2e-aws-rhcos4-stig
  • /test 4.15-e2e-rosa-ocp4-cis-node
  • /test 4.15-e2e-rosa-ocp4-pci-dss-node
  • /test 4.15-images
  • /test 4.16-e2e-aws-ocp4-bsi
  • /test 4.16-e2e-aws-ocp4-bsi-node
  • /test 4.16-e2e-aws-ocp4-cis
  • /test 4.16-e2e-aws-ocp4-cis-node
  • /test 4.16-e2e-aws-ocp4-e8
  • /test 4.16-e2e-aws-ocp4-high
  • /test 4.16-e2e-aws-ocp4-high-node
  • /test 4.16-e2e-aws-ocp4-moderate
  • /test 4.16-e2e-aws-ocp4-moderate-node
  • /test 4.16-e2e-aws-ocp4-pci-dss
  • /test 4.16-e2e-aws-ocp4-pci-dss-node
  • /test 4.16-e2e-aws-ocp4-stig
  • /test 4.16-e2e-aws-ocp4-stig-node
  • /test 4.16-e2e-aws-rhcos4-bsi
  • /test 4.16-e2e-aws-rhcos4-e8
  • /test 4.16-e2e-aws-rhcos4-high
  • /test 4.16-e2e-aws-rhcos4-moderate
  • /test 4.16-e2e-aws-rhcos4-stig
  • /test 4.16-images
  • /test 4.17-e2e-aws-ocp4-bsi
  • /test 4.17-e2e-aws-ocp4-bsi-node
  • /test 4.17-e2e-aws-ocp4-cis
  • /test 4.17-e2e-aws-ocp4-cis-node
  • /test 4.17-e2e-aws-ocp4-e8
  • /test 4.17-e2e-aws-ocp4-high
  • /test 4.17-e2e-aws-ocp4-high-node
  • /test 4.17-e2e-aws-ocp4-moderate
  • /test 4.17-e2e-aws-ocp4-moderate-node
  • /test 4.17-e2e-aws-ocp4-pci-dss
  • /test 4.17-e2e-aws-ocp4-pci-dss-node
  • /test 4.17-e2e-aws-ocp4-stig
  • /test 4.17-e2e-aws-ocp4-stig-node
  • /test 4.17-e2e-aws-rhcos4-bsi
  • /test 4.17-e2e-aws-rhcos4-e8
  • /test 4.17-e2e-aws-rhcos4-high
  • /test 4.17-e2e-aws-rhcos4-moderate
  • /test 4.17-e2e-aws-rhcos4-stig
  • /test 4.17-images
  • /test e2e-aws-ocp4-bsi
  • /test e2e-aws-ocp4-bsi-node
  • /test e2e-aws-ocp4-cis
  • /test e2e-aws-ocp4-cis-node
  • /test e2e-aws-ocp4-e8
  • /test e2e-aws-ocp4-high
  • /test e2e-aws-ocp4-high-node
  • /test e2e-aws-ocp4-moderate
  • /test e2e-aws-ocp4-moderate-node
  • /test e2e-aws-ocp4-pci-dss
  • /test e2e-aws-ocp4-pci-dss-node
  • /test e2e-aws-ocp4-stig
  • /test e2e-aws-ocp4-stig-node
  • /test e2e-aws-rhcos4-bsi
  • /test e2e-aws-rhcos4-e8
  • /test e2e-aws-rhcos4-high
  • /test e2e-aws-rhcos4-moderate
  • /test e2e-aws-rhcos4-stig
  • /test images

Use /test all to run the following jobs that were automatically triggered:

  • pull-ci-ComplianceAsCode-content-master-4.12-images
  • pull-ci-ComplianceAsCode-content-master-4.13-images
  • pull-ci-ComplianceAsCode-content-master-4.14-images
  • pull-ci-ComplianceAsCode-content-master-4.15-images
  • pull-ci-ComplianceAsCode-content-master-4.16-images
  • pull-ci-ComplianceAsCode-content-master-4.17-images
  • pull-ci-ComplianceAsCode-content-master-images

In response to this:

/test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

openshift-ci[bot] avatar Aug 09 '24 17:08 openshift-ci[bot]

/test 4.12-e2e-aws-ocp4-cis /test 4.13-e2e-aws-ocp4-cis /test 4.14-e2e-aws-ocp4-cis /test 4.15-e2e-aws-ocp4-cis /test 4.16-e2e-aws-ocp4-cis

rhmdnd avatar Aug 09 '24 17:08 rhmdnd

verification pass with ghcr.io/complianceascode/k8scontent:12220 + 4.17.0-0.nightly-2024-08-09-031511:

1. Create a ssb:
 % cat ssb_cis_p.yaml 
apiVersion: compliance.openshift.io/v1alpha1
kind: ScanSettingBinding
metadata:
  name: cis-compliance
  namespace: openshift-compliance
profiles:
  - name: upstream-ocp4-cis
    kind: Profile
    apiGroup: compliance.openshift.io/v1alpha1
settingsRef:
  name: default-auto-apply
  kind: ScanSetting
  apiGroup: compliance.openshift.io/v1alpha1
 % oc apply -f ssb_cis_p.yaml 
scansettingbinding.compliance.openshift.io/cis-compliance created
% oc get suite -w
NAME             PHASE     RESULT
cis-compliance   RUNNING   NOT-AVAILABLE
cis-compliance   AGGREGATING   NOT-AVAILABLE
cis-compliance   DONE          NON-COMPLIANT
cis-compliance   DONE          NON-COMPLIANT
^C                                                                                                                                                                                                               % oc get cr
NAME                                                                      STATE
upstream-ocp4-cis-api-server-encryption-provider-cipher                   Applied
upstream-ocp4-cis-audit-profile-set                                       Applied
upstream-ocp4-cis-kubelet-configure-tls-cipher-suites-ingresscontroller   Applied
 % oc -n openshift-ingress-operator get ingresscontroller/default -o=jsonpath={.spec.tlsSecurityProfile.custom.ciphers} | jq -r
[
  "ECDHE-ECDSA-AES128-GCM-SHA256",
  "ECDHE-RSA-AES128-GCM-SHA256",
  "ECDHE-ECDSA-CHACHA20-POLY1305",
  "ECDHE-RSA-AES256-GCM-SHA384",
  "ECDHE-RSA-CHACHA20-POLY1305",
  "ECDHE-ECDSA-AES256-GCM-SHA384",
  "TLS_AES_128_GCM_SHA256",
  "TLS_AES_256_GCM_SHA384",
  "TLS_AES_128_CCM_SHA256",
  "TLS_CHACHA20_POLY1305_SHA256"
]
2. Trigger a rescan and check the ccr status after rescan:
% oc annotate compliancescan/upstream-ocp4-cis compliance.openshift.io/rescan=
compliancescan.compliance.openshift.io/upstream-ocp4-cis annotated
% oc get suite -w
NAME             PHASE     RESULT
cis-compliance   RUNNING   NOT-AVAILABLE
cis-compliance   AGGREGATING   NOT-AVAILABLE
cis-compliance   DONE          NON-COMPLIANT
cis-compliance   DONE          NON-COMPLIANT
^C%                                                                                                                                                                                                                 % oc get ccr upstream-ocp4-cis-kubelet-configure-tls-cipher-suites-ingresscontroller --show-labels
NAME                                                                      STATUS   SEVERITY   LABELS
upstream-ocp4-cis-kubelet-configure-tls-cipher-suites-ingresscontroller   PASS     medium     compliance.openshift.io/automated-remediation=,compliance.openshift.io/check-severity=medium,compliance.openshift.io/check-status=PASS,compliance.openshift.io/profile-guid=a230315d-3e4a-5b58-b00f-f96f1553e036,compliance.openshift.io/scan-name=upstream-ocp4-cis,compliance.openshift.io/suite=cis-compliance
% oc get ccr -l compliance.openshift.io/automated-remediation=,compliance.openshift.io/check-status=FAIL
No resources found in openshift-compliance namespace.

xiaojiey avatar Aug 12 '24 10:08 xiaojiey

/label qe-approved

xiaojiey avatar Aug 12 '24 10:08 xiaojiey

@xiaojiey: The label(s) qe-approved cannot be applied, because the repository doesn't have them.

In response to this:

/label qe-approved

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

openshift-ci[bot] avatar Aug 12 '24 10:08 openshift-ci[bot]

/unhold

xiaojiey avatar Aug 12 '24 10:08 xiaojiey

@yuumasato @Vincent056 this should be ready for another look

rhmdnd avatar Aug 12 '24 13:08 rhmdnd

Sorry for the back and forth here. I checked https://github.com/openshift/cluster-ingress-operator/blob/7cf86c7c749fa62b9cffd9a62275e33a4083985e/manifests/00-custom-resource-definition.yaml#L1712-L1716, seems "TLS_AES_128_CCM_SHA256" cipher was not recommended here. I am not sure whether we need to keep it in the custom list or not.

xiaojiey avatar Aug 13 '24 03:08 xiaojiey

I am not sure whether we need to keep it in the custom list or not.

I can remove it. I found it in https://ciphersuite.info/cs/TLS_AES_128_CCM_SHA256/ which says it's secure, not recommended. We can add it to the rule later if needed, but the other updates we've made are useful on their own and align better than what we have today.

rhmdnd avatar Aug 13 '24 13:08 rhmdnd

/test 4.12-e2e-aws-ocp4-cis /test 4.13-e2e-aws-ocp4-cis /test e2e-aws-ocp4-cis /test 4.15-e2e-aws-ocp4-cis /test 4.16-e2e-aws-ocp4-cis

rhmdnd avatar Aug 13 '24 13:08 rhmdnd

Still debugging some of the issues with the remediation. Looks like the e2e tests are pointing out a legitimate issue.

rhmdnd avatar Aug 13 '24 22:08 rhmdnd

/test 4.12-e2e-aws-ocp4-cis /test 4.13-e2e-aws-ocp4-cis /test e2e-aws-ocp4-cis /test 4.15-e2e-aws-ocp4-cis /test 4.16-e2e-aws-ocp4-cis

rhmdnd avatar Aug 14 '24 22:08 rhmdnd

Code Climate has analyzed commit 3c24d289 and detected 0 issues on this pull request.

The test coverage on the diff in this pull request is 100.0% (50% is the threshold).

This pull request will bring the total coverage in the repository to 59.4% (0.0% change).

View more on Code Climate.

qlty-cloud-legacy[bot] avatar Aug 14 '24 23:08 qlty-cloud-legacy[bot]

/test 4.12-e2e-aws-ocp4-cis /test 4.13-e2e-aws-ocp4-cis /test e2e-aws-ocp4-cis /test 4.15-e2e-aws-ocp4-cis /test 4.16-e2e-aws-ocp4-cis

Underlying cluster issues unrelated to this change.

rhmdnd avatar Aug 15 '24 00:08 rhmdnd

The remediations applied through the test suite does cause the cluster operators to update. Checking to see if waiting for all operator to come back as stable affects this PR.

https://github.com/ComplianceAsCode/ocp4e2e/pull/51

rhmdnd avatar Aug 21 '24 21:08 rhmdnd