Fabric Orderers prevent rotation of TLS certificates

[celder628]

Fabric orderers 2.2.x and 2.4.x prevent the rotation of TLS certificates in the code:

orderer/consensus/etcdraft/chain.go

	active := c.ActiveNodes.Load().([]uint64)
	if changes.UnacceptableQuorumLoss(active) {
		return errors.Errorf("%d out of %d nodes are alive, configuration will result in quorum loss", len(active), len(dummyOldConsentersMap))
	}

Processes that would require this step would include migration of a set of orderers to a new environment with a new host name.

The new hostname in the SAN will invalidate the existing TLS certs. This requires updating the TLS certs one at a time and forcing a new leader election and a new quorum on the new TLS certs.

The check added in the 2.2 stream will block the changeover.

Example. If there are 3 ordering node. When the attempt is made to update the channel on the second ordering node, this check will detect a loss of quorum and block the change.

Recommendation would be to add a orderer.yaml configuration item to allow an override for this check.

[yacovm]

I'm not sure I follow, you want to be able to rotate the certificates of more than a single orderer at a time? why?

[denyeart]

My suggestion was to 'reenroll' for each TLS cert in fabric-ca with the same existing private key but provide the new SAN in the reenroll request using --csr.hosts. When the same key is used, there should be no need to do a channel config update. Would this approach work?

[celder628]

If the private key is the same, you are right. The channel does not necessarily need to be updated.

Here is the other scenario.

You have created TLS certificates with an external CA: OverconfidentStartup.com

OverconfidentStartup.com then announces that their service is ceasing operation effective immediately.

There would need to be a way to migrate the ordering nodes one at a time to a new CA.

[celder628]

Channel updates are still required in the case of moving clusters. The cert should be valid since the private key is unchanged.

The server name still has to be changed in the channel. Example:

                "consenters": [
                  {
                    "client_tls_cert": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM2akNDQXBDZ0F3SUJBZ0lVTERKdFV3MEVSYXUwMi9raVd0dkczcVd4bEk4d0NnWUlLb1pJemowRUF3SXcKYURFTE1Ba0dBMVVFQmhNQ1ZWTXhGekFWQmdOVkJBZ1REazV2Y25Sb0lFTmhjbTlzYVc1aE1SUXdFZ1lEVlFRSwpFd3RJZVhCbGNteGxaR2RsY2pFUE1BMEdBMVVFQ3hNR1JtRmljbWxqTVJrd0Z3WURWUVFERXhCdmNtUmxjbVZ5Ck1XTmhMWFJzYzJOaE1CNFhEVEl6TURJeE5USXdNVEF3TUZvWERUTTRNREl4TVRJd01UQXdNRm93SlRFUU1BNEcKQTFVRUN4TUhiM0prWlhKbGNqRVJNQThHQTFVRUF4TUliM0prWlhKbGNqRXdXVEFUQmdjcWhrak9QUUlCQmdncQpoa2pPUFFNQkJ3TkNBQVE0MlZlNjQ0R1NhWkFyd0RPUU44TElTY2FWaXBPMXkzQjh6Rm1CUVFtdW1HYXFQVStMCnNsR2wweFFacU5HczJaTUNsZU9vbEY4dWNNeU9LQVVKMlNWeG80SUJXVENDQVZVd0RnWURWUjBQQVFIL0JBUUQKQWdPb01CMEdBMVVkSlFRV01CUUdDQ3NHQVFVRkJ3TUJCZ2dyQmdFRkJRY0RBakFNQmdOVkhSTUJBZjhFQWpBQQpNQjBHQTFVZERnUVdCQlN0RVUvWWtFUU1FTXFkNUdZTnc0UGxwY1ZaOGpBZkJnTlZIU01FR0RBV2dCU3krbis5CjFQMG0yVHVxa3NoLzNlTWFTc3F0M2pCNEJnTlZIUkVFY1RCdmdtZHVNV1k1TW1GakxXOXlaR1Z5WlhJeGJtOWsKWlRFdVkyVnNaR1Z5TlMwek16UmxNVGxpTlRZek5EZGtPV05sTXpKaU5tUTJZVGczTUdReE5HWXpOeTB3TURBdwpMblZ6TFhOdmRYUm9MbU52Ym5SaGFXNWxjbk11WVhCd1pHOXRZV2x1TG1Oc2IzVmtod1IvQUFBQk1Gd0dDQ29ECkJBVUdCd2dCQkZCN0ltRjBkSEp6SWpwN0ltaG1Ma0ZtWm1sc2FXRjBhVzl1SWpvaUlpd2lhR1l1Ulc1eWIyeHMKYldWdWRFbEVJam9pYjNKa1pYSmxjakVpTENKb1ppNVVlWEJsSWpvaWIzSmtaWEpsY2lKOWZUQUtCZ2dxaGtqTwpQUVFEQWdOSUFEQkZBaUVBMkVCaUZkbnYycmFxOG5JSG5RRHVsU1JMU01kbUI2cm5qa2ladDdiWWJGb0NJRkZnCjBCbkFIL1VLRldQVmdxU0JyWDhqZHQ4NjdWSE11UVZCZEt1KzNhbFcKLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=",
                    "host": "n1f92ac-orderer1node1.celder5-334e19b56347d9ce32b6d6a870d14f37-0000.us-south.containers.appdomain.cloud",
                    "port": 7050,
                    "server_tls_cert": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM2akNDQXBDZ0F3SUJBZ0lVTERKdFV3MEVSYXUwMi9raVd0dkczcVd4bEk4d0NnWUlLb1pJemowRUF3SXcKYURFTE1Ba0dBMVVFQmhNQ1ZWTXhGekFWQmdOVkJBZ1REazV2Y25Sb0lFTmhjbTlzYVc1aE1SUXdFZ1lEVlFRSwpFd3RJZVhCbGNteGxaR2RsY2pFUE1BMEdBMVVFQ3hNR1JtRmljbWxqTVJrd0Z3WURWUVFERXhCdmNtUmxjbVZ5Ck1XTmhMWFJzYzJOaE1CNFhEVEl6TURJeE5USXdNVEF3TUZvWERUTTRNREl4TVRJd01UQXdNRm93SlRFUU1BNEcKQTFVRUN4TUhiM0prWlhKbGNqRVJNQThHQTFVRUF4TUliM0prWlhKbGNqRXdXVEFUQmdjcWhrak9QUUlCQmdncQpoa2pPUFFNQkJ3TkNBQVE0MlZlNjQ0R1NhWkFyd0RPUU44TElTY2FWaXBPMXkzQjh6Rm1CUVFtdW1HYXFQVStMCnNsR2wweFFacU5HczJaTUNsZU9vbEY4dWNNeU9LQVVKMlNWeG80SUJXVENDQVZVd0RnWURWUjBQQVFIL0JBUUQKQWdPb01CMEdBMVVkSlFRV01CUUdDQ3NHQVFVRkJ3TUJCZ2dyQmdFRkJRY0RBakFNQmdOVkhSTUJBZjhFQWpBQQpNQjBHQTFVZERnUVdCQlN0RVUvWWtFUU1FTXFkNUdZTnc0UGxwY1ZaOGpBZkJnTlZIU01FR0RBV2dCU3krbis5CjFQMG0yVHVxa3NoLzNlTWFTc3F0M2pCNEJnTlZIUkVFY1RCdmdtZHVNV1k1TW1GakxXOXlaR1Z5WlhJeGJtOWsKWlRFdVkyVnNaR1Z5TlMwek16UmxNVGxpTlRZek5EZGtPV05sTXpKaU5tUTJZVGczTUdReE5HWXpOeTB3TURBdwpMblZ6TFhOdmRYUm9MbU52Ym5SaGFXNWxjbk11WVhCd1pHOXRZV2x1TG1Oc2IzVmtod1IvQUFBQk1Gd0dDQ29ECkJBVUdCd2dCQkZCN0ltRjBkSEp6SWpwN0ltaG1Ma0ZtWm1sc2FXRjBhVzl1SWpvaUlpd2lhR1l1Ulc1eWIyeHMKYldWdWRFbEVJam9pYjNKa1pYSmxjakVpTENKb1ppNVVlWEJsSWpvaWIzSmtaWEpsY2lKOWZUQUtCZ2dxaGtqTwpQUVFEQWdOSUFEQkZBaUVBMkVCaUZkbnYycmFxOG5JSG5RRHVsU1JMU01kbUI2cm5qa2ladDdiWWJGb0NJRkZnCjBCbkFIL1VLRldQVmdxU0JyWDhqZHQ4NjdWSE11UVZCZEt1KzNhbFcKLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo="
                  },

[tock-ibm]

You have created TLS certificates with an external CA: OverconfidentStartup.com

You have to first add the new CA the new certs are coming from, then update the orderers one by one, never losing quorum. After all orderers where updated, remove the old CA.