Ignite cluster (both client and server) breaking intermittently with 401 http code in AKS

[kaynat12]

I am able to deploy ignite cluster in AKS successfully

Note: 1. service account, role and role binding are created already and have also mentioned service account in deployment.yaml 2. service name and namespace are also mentioned in ignite configuration 3. 3. We have configured thin clients in cluster

Client and server are able to join and communicate initially but after below intermittent exception, both are unable to reconnect to cluster and unfortunately, we have to redeploy entire helm chart,

org.apache.ignite.logger.java.JavaLogger error SEVERE: Failed to clean IP finder up class org.apache.ignite.spi.IgniteSpiException: Failed to retrieve Ignite pods IP addresses. at org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder.getRegisteredAddresses(TcpDiscoveryKubernetesIpFinder.java:81) at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.registeredAddresses(TcpDiscoverySpi.java:2057) at org.apache.ignite.spi.discovery.tcp.ServerImpl$IpFinderCleaner.cleanIpFinder(ServerImpl.java:2279) Caused by: class org.apache.ignite.IgniteException: Failed to retrieve Ignite pods IP addresses. at org.apache.ignite.internal.kubernetes.connection.KubernetesServiceAddressResolver.getServiceAddresses(KubernetesServiceAddressResolver.java:123) Caused by: java.io.IOException: Server returned HTTP response code: 401 for URL: https://kubernetes.default.svc.cluster.local:443/api/v1/namespaces/ignite/endpoints/cache-service at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:2000) at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1589) at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:224) at org.apache.ignite.internal.kubernetes.connection.KubernetesServiceAddressResolver.getServiceAddresses(KubernetesServiceAddressResolver.java:111)

[timoninmaxim]

Hi, 401 is Unauthorized exception. By default Ignite uses account token located in "/var/run/secrets/kubernetes.io/serviceaccount/token". Custom can be set with KubernetesConnectionConfiguration#setAccountToken. Please check your accounts configuration.