When starting up two is servers by setting replica 2, DUPLICATE entry execption is thrown.

[Megala21]

Description: When starting up two is servers by setting replica 2, DUPLICATE entry execption is thrown. This exception is thrown due to two nodes trying to write the same entry simultaneously.

Ex - DUPLICATE_ENTRY exception when creating admin user, admin role.

Suggested Labels: Bug

Suggested Assignees: N/A

Affected Product Version: IS 5.4.0, 5.4.1, 5.5.0

OS, DB, other environment details and versions: Tested aginnst google cloud kubernates engine.

Steps to reproduce: Add replica count 2, in is-deployment.yaml and deploy and see the pod logs.

Related Issues: N/A

[chirangaalwis]

@Megala21 @malithie I experienced the same problem a number of times.

A possible solution for this is the introduction of Kubernetes liveness probes [1].

For example, under the WSO2 product Kubernetes resource unification process that began recently, I managed to build up a IS two node cluster in which I've used liveness probes to conduct port scanning (port 9443) to check the liveness of each IS pod as follows:

        livenessProbe:
          exec:
            command:
            - /bin/bash
            - -c
            - nc -z localhost 9443
          initialDelaySeconds: 100
          periodSeconds: 10

If the server startup has failed due to the discussed issue, the liveness probe fails. As a result, it will spawn up a new IS pod after killing the existing unhealthy pod.

[1]: Kubernetes liveness and readiness probes

[chirangaalwis]

During testing it was experienced that the solution suggested in https://github.com/wso2/kubernetes-is/issues/7#issuecomment-392869548 does not provide the most perfect solution to the discussed issue. It was experienced that the server startup takes place successfully even while the above errors are occurring.

Hence, the current solution that has been adopted is to instruct the user to scale [1] the number of Identity Server instances as desired. By default, the setup is initiated with a single Identity Server instance.

[1]: Instruction to scale the number of nodes

[chirangaalwis]

@malithie, @omindu shall we close this, if this issue has been tested and fixed in latest Identity Server versions?

[senthalan]

Still, we have errors when starting the server parallel. refer : https://github.com/wso2/product-is/issues/5936