solana
solana copied to clipboard
solana-labs
.
@JonathanStables Which build did you actually reproduce the issue?
1.13.2 and 1.13.3
Just to add to this issue we see it on v1.13.3 as well, and immediately after getting all the behavior described above we see also CPU time going up for the tokyo-runtime-rw
Here are some quic stats at the time
Having enabled full-rpc-api on our staked validator last Friday we saw a massive spike in skip rate, with about 30% of leader slots being skipped. After disabling full-rpc-api we resumed normal skip rate.
We did not advertise the RPC port in gossip and it was exposed to a single sender via IP whitelist only, so it seems more likely just having the flag set rather than anything else triggering it, or at most low level RPC activity.
Having enabled full-rpc-api on our staked validator last Friday we saw a massive spike in skip rate, with about 30% of leader slots being skipped. After disabling full-rpc-api we resumed normal skip rate.
We did not advertise the RPC port in gossip and it was exposed to a single sender via IP whitelist only, so it seems more likely just having the flag set rather than anything else triggering it, or at most low level RPC activity.
How does it relate to the QUIC issue?
How does it relate to the QUIC issue?
Perhaps I misunderstood the implications of this bug but given the connection between enabling RPC and extremely high skip rate and seeing this issue my first thought is that enabling RPC is somehow affecting the validator's ability to transmit it's shreds timeously.
Hello, same problem for me:
solana-validator 1.13.5 (src:91384943; feat:1365939126) Firewall is ok. Node is in sync. quic enabled (default settings)
But i can not make the transactions at all:
"[2022-12-20T08:18:03.368775720Z INFO solana_send_transaction_service::send_transaction_service] Dropping expired transaction: ..."
I will be happy if someone advice me where to look.
Update: i was able to make the transaction using other node which have quic disabled and UDP enabled. validator version is the same.
Update2: pub const VALIDATOR_PORT_RANGE: PortRange = (8000, 10_000);
Seems that --dynamic-port-range is ignored in this case. So our solution was to enable this particular range in FW.
