feat(core/config): enable override of rpc.host and grpc.host, and add ability to use secure connections

[ramin]

• builds upon but largely replaces https://github.com/celestiaorg/celestia-node/pull/3153 and totally replaces https://github.com/celestiaorg/celestia-node/pull/3154
• enabled splitting of RPC config and GRPC config so we can set unique hosts AND ports for each
• modifies concept of IP to Host for the new configs (ie: core.grpc.host instead of core.grpc.ip as it just makes more sense as we are using DNS)
• preserves the core.ip so it doesn't force double configuration for now and a hard break (though maybe we want that?)
• also introduces ability to override the default http schema, modifying the url validation to expect a host with a scheme and falling back to http, so we can make http and https requests to core over RPC and grpc
• also adds (highly untested) ability to set a grpc cert for mutual tls (@Bidon15 to work with me on testing and verifying)

[Bidon15]

running into issues

CLI command to start:

celestia light start --p2p.network mocha --core.grpc.host grpc-mocha.pops.one --core.rpc.host https://rpc-mocha.pops.one/ --core.rpc.port 443

Then I try to celestia state account-address --token $CEL_AUTH_TOKEN Which produces the following error:

2024-03-06T14:33:29.155+0100	WARN	rpc	[email protected]/handler.go:444	error in RPC call to 'state.AccountAddress': node is running without state access. run with --core.ip <CORE NODE IP> to resolve

[Bidon15]

$ celestia light start --p2p.network mocha --core.grpc.host grpc-mocha.pops.one --core.rpc.host full.consensus.mocha-4.celestia-mocha.com
$ export CEL_AUTH_TOKEN=$(celestia light auth admin --p2p.network mocha)
$ celestia state account-address --token $CEL_AUTH_TOKEN

both endpoints are not secured and pointing to 9090 and 26657 respectively

$ celestia light start --p2p.network mocha --core.grpc.host grpc.celestia-mocha.com --core.grpc.port 443 --core.rpc.host https://rpc-mocha.pops.one --core.rpc.port 443
$ export CEL_AUTH_TOKEN=$(celestia light auth admin --p2p.network mocha)
$ celestia state account-address --token $CEL_AUTH_TOKEN

both endpoints are secured here

[Bidon15]

celestia light start --p2p.network mocha --core.grpc.host consensus-full-mocha-4.celestia-mocha.com --core.rpc.host rpc-mocha.pops.one --core.rpc.port 443 --core.rpc.https

# or 

celestia light start --p2p.network mocha --core.grpc.host consensus-full-mocha-4.celestia-mocha.com --core.rpc.host rpc.celestia-mocha.com --core.rpc.port 443 --core.rpc.https

produces these errors

2024-04-03T16:01:36.872+0200	WARN	rpc	[email protected]/handler.go:444	error in RPC call to 'state.Balance': post failed: Post "https://46.166.170.198:443": tls: failed to verify certificate: x509: cannot validate certificate for 46.166.170.198 because it doesn't contain any IP SANs

2024-04-03T16:02:30.759+0200	WARN	rpc	[email protected]/handler.go:444	error in RPC call to 'state.Balance': post failed: Post "https://151.115.15.15:443": tls: failed to verify certificate: x509: certificate signed by unknown authority

[ramin]

@Bidon15 try like this:

celestia light start --p2p.network mocha --core.grpc.host grpc-mocha.pops.one --core.rpc.host full.consensus.mocha-4.celestia-mocha.com --core.rpc.https

so flags as

--p2p.network mocha --core.grpc.host grpc-mocha.pops.one --core.rpc.host full.consensus.mocha-4.celestia-mocha.com --core.rpc.https

sets two separate hosts (1 for grpc, one for rpc) and then allows https for RPC

lets pair and get this over line tomorrow

[Bidon15]

@ramin

celestia light start --p2p.network mocha --core.grpc.host grpc-mocha.pops.one --core.rpc.host rpc.celestia-mocha.com --core.rpc.https --core.rpc.port 443

produces this error

celestia state balance --token $CEL_AUTH_TOKEN --node.store ~/.celestia-light-mocha-4/
{
  "result": "post failed: Post \"https://151.115.15.15:443\": tls: failed to verify certificate: x509: certificate signed by unknown authority"
}

as an example that the rpc works

curl -X GET "https://rpc.celestia-mocha.com:443/status" -H "accept: application/json"
{"jsonrpc":"2.0","id":-1,"result":{"node_info":{"protocol_version":{"p2p":"8","block":"11","app":"1"},"id":"a6808abd424bce099299b921bfc053721064b9a3","listen_addr":"tcp://0.0.0.0:26656","network":"mocha-4","version":"1.7.0","channels":"40202122233038606100","moniker":"consensus-full-1-0","other":{"tx_index":"on","rpc_address":"tcp://0.0.0.0:26657"}},"sync_info":{"latest_block_hash":"666C946F731FB5716BEE6579D8A6340029D7056DAEF70DDDD79C7D22A445F492","latest_app_hash":"4E6808C60F654DC0DA6B4220E2D0A5468DC9D11B3CFC1CB08F49DCE413BD8C18","latest_block_height":"1663999","latest_block_time":"2024-04-22T09:37:45.451761215Z","earliest_block_hash":"B93BBE20A0FBFDF955811B6420F8433904664D45DB4BF51022BE4200C1A1680D","earliest_app_hash":"E3B0C44298FC1C149AFBF4C8996FB92427AE41E4649B934CA495991B7852B855","earliest_block_height":"1","earliest_block_time":"2023-09-06T03:15:51.510579711Z","catching_up":false},"validator_info":{"address":"765D4BC78000385164297594BEAB9341738C9751","pub_key":{"type":"tendermint/PubKeyEd25519","value":"s8TABsDmpPblLg8XMlCGiLwzIyRJoZAzTRmHa/q5VGs="},"voting_power":"0"}}}%     

[liamsi]

as an example that the rpc works

@Bidon15 your curl command does not work for me either. but looking at the error you shared this indicates that node is trying to lookup the cert for the IP while your curl command uses a valid domain-name. Your browser would or at least also should complain about https://151.115.15.15/ while it would not for https://rpc.celestia-mocha.com

[renaynay]

@Bidon15 Do you still want this PR for BNs ? We will be removing need for rpc flag for FNs and LNs in v0.14.0, but likely not for BNs in that release.

[Bidon15]

@Bidon15 Do you still want this PR for BNs ? We will be removing need for rpc flag for FNs and LNs in v0.14.0, but likely not for BNs in that release.

Hey @renaynay. Fine for me if we achieve the following result:

1. FNs and LNs only have --core.grpc && --core.grpc.port flags
2. BNs have both --core.grpc && --core.rpc flags and ports flags, too

Reasoning: when we ship https://github.com/celestiaorg/celestia-node/issues/3295, most RaaS teams will utilise a single DA BN for their EEs

[renaynay]

@Bidon15 So should we close this PR?

[Bidon15]

Yes, the follow-up is here https://github.com/celestiaorg/celestia-node/pull/3359