istio
Seeing DNS i/o timeout sporadically 0.1% in a day
Is this the right place to submit this?
- [X] This is not a security vulnerability or a crashing bug
- [X] This is not a question about how to use Istio
Bug Description
We are seeing the sporadic i/o timeout for DNS querries , When we enable the debug log for istio-proxy sidecar we see SERVFAIL at the same time. This is not specific any endpoint or A/AAAA record . We tried all the possibilties of configuration on coredns side like cache 60, cache { servfail 0 } , buffersize nothing seems to be fixing the issue.
Just is this error thrown by sidecar when dns reaquest struck more than a 5sec ? Is there any way we can debug the issue.
upstream failure: read udp 10.2.166.81:36064->172.20.0.10:53: i/o timeout
debug log says SERVFAIL :
upstream response for hostname "ocsp2.globalsign.com.core.svc.cluster.local." : ;; opcode: QUERY, status: SERVFAIL, id: 10321 ;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;ocsp2.globalsign.com.core.svc.cluster.local. IN A
--
Version
$ kubectl version
Client Version: v1.28.2
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.27.13-eks-3af4770
$ istioctl version
client version: 1.20.3
control plane version: 1.20.3
data plane version: 1.20.3 (1187 proxies)
Additional Information
No response
Affected product area
- [ ] Ambient
- [ ] Docs
- [ ] Dual Stack
- [ ] Installation
- [X] Networking
- [x] Performance and Scalability
- [ ] Extensions and Telemetry
- [ ] Security
- [ ] Test and Release
- [ ] User Experience
- [ ] Developer Infrastructure
- [ ] Upgrade
- [x] Multi Cluster
- [ ] Virtual Machine
- [ ] Control Plane Revisions
No logs on the coredns During the timeout error timeframe . And We see issues on those clusters where we enable smart-dns.
🚧 This issue or pull request has been closed due to not having had activity from an Istio team member since 2024-07-13. If you feel this issue or pull request deserves attention, please reopen the issue. Please see this wiki page for more information. Thank you for your contributions.
Created by the issue and PR lifecycle manager.
