neofs-node
neofs-node copied to clipboard
tree: Replication request fails due to gRPC/TCP closed connection when node under load
when NeoFS storage nodes are under high load, Tree service's replication sometimes fail:
Jul 13 14:47:32 neofs-node[551040]: 2023-07-13T14:47:32.550Z warn tree/replicator.go:85 failed to sent update to the node {"last_error": "rpc error: code = Unavailable desc = error reading from server: read tcp [<ipv6>]:53262->[<ipv6>]:8080: use of closed network connection", "address": "/dns4/dns/tcp/8080](http://dns/tcp/8080)", "key": "<key>"}
seems like used gRPC conn bases on already closed TCP conn, but it's not obvious what closed it and why
- being Tree service client, NeoFS S3 Gateway encounters similar problem during load https://github.com/nspcc-dev/neofs-s3-gw/issues/797