cli icon indicating copy to clipboard operation
cli copied to clipboard

feat: remove `protoc` pkg and also nodetime helpers `ts-proto` and `sta`

Open Pantani opened this issue 10 months ago • 9 comments

Description

stop using the protoc with the node plugins so we can remove the node time binary. This PR's makes all ts generation using the buf generate command.

How to test

ignite s chain venus && cd venus && ignite g ts-client

check the ts-client folder into the chain

Pantani avatar Apr 20 '24 00:04 Pantani

How do we provide discovered modules' proto files into code generation process?

is almost the same thing as the protoc, we are passing the discovered modules to the include-import flag into the buf generation

Pantani avatar Apr 20 '24 10:04 Pantani

Tried testing off this branch:

Chain scaffolds ok..

On ignite g ts-client, I get this prompt:

Now ignite supports the `buf.build` (https://buf.build) registry to manage the protobuf dependencies. The embed protoc binary was deprecated and, your blockchain is still using it. Would you like to upgrade and add the `buf.build` config files to `proto/` folder: y/N

Answering No:

✘ Code generation using protoc is only supported by Ignite CLI v0.26.1 or older

Answering Yes:

 buf.build files added: 

/media/clockwork/Projects/Tendermint/venus//buf.gen.ts.yaml
/media/clockwork/Projects/Tendermint/venus//buf.lock
/media/clockwork/Projects/Tendermint/venus//buf.yaml
/media/clockwork/Projects/Tendermint/venus//buf.gen.gogo.yaml
/media/clockwork/Projects/Tendermint/venus//buf.gen.pulsar.yaml
/media/clockwork/Projects/Tendermint/venus//buf.gen.sta.yaml
/media/clockwork/Projects/Tendermint/venus//buf.gen.swagger.yaml

✗ Ignite proto directory path from the chain config doesn't match the proto directory path from the chain `buf.work.yaml`. Do you want to add the proto path `` to the directories list from the buf work file Y/N:

Answering Yes:

✘ Cannot build app:                                                                                                                                                                                                                                                                                                                                                                                                                               
                                                                                                                                                                                                                                                                                                                                                                                                                                                
error while running command /home/clockwork/go/bin/buf export /media/clockwork/Projects/Tendermint/venus/proto --output=/tmp/includeFolder1117887864: Failure: directory "." listed in /media/clockwork/Projects/Tendermint/venus/buf.work.yaml, it is not valid to have "." as a workspace directory, as this is no different than not having a workspace at all, see https://buf.build/docs/reference/workspaces/#directories for more details
: exit status 1

Answering No:

✘ Cannot build app:                                                                                   
                                                                                                    
invalid Cosmos SDK mod path: [/home/clockwork/go/pkg/mod/github.com/cosmos/[email protected]/proto]

Guessing we need to explicitly supply the proto dir? or the default is not picked up correctly in ignite g commands?

clockworkgr avatar Apr 23 '24 09:04 clockworkgr

Tried testing off this branch:

Chain scaffolds ok..

On ignite g ts-client, I get this prompt:

Now ignite supports the `buf.build` (https://buf.build) registry to manage the protobuf dependencies. The embed protoc binary was deprecated and, your blockchain is still using it. Would you like to upgrade and add the `buf.build` config files to `proto/` folder: y/N

Answering No:

✘ Code generation using protoc is only supported by Ignite CLI v0.26.1 or older

Answering Yes:

 buf.build files added: 

/media/clockwork/Projects/Tendermint/venus//buf.gen.ts.yaml
/media/clockwork/Projects/Tendermint/venus//buf.lock
/media/clockwork/Projects/Tendermint/venus//buf.yaml
/media/clockwork/Projects/Tendermint/venus//buf.gen.gogo.yaml
/media/clockwork/Projects/Tendermint/venus//buf.gen.pulsar.yaml
/media/clockwork/Projects/Tendermint/venus//buf.gen.sta.yaml
/media/clockwork/Projects/Tendermint/venus//buf.gen.swagger.yaml

✗ Ignite proto directory path from the chain config doesn't match the proto directory path from the chain `buf.work.yaml`. Do you want to add the proto path `` to the directories list from the buf work file Y/N:

Answering Yes:

✘ Cannot build app:                                                                                                                                                                                                                                                                                                                                                                                                                               
                                                                                                                                                                                                                                                                                                                                                                                                                                                
error while running command /home/clockwork/go/bin/buf export /media/clockwork/Projects/Tendermint/venus/proto --output=/tmp/includeFolder1117887864: Failure: directory "." listed in /media/clockwork/Projects/Tendermint/venus/buf.work.yaml, it is not valid to have "." as a workspace directory, as this is no different than not having a workspace at all, see https://buf.build/docs/reference/workspaces/#directories for more details
: exit status 1

Answering No:

✘ Cannot build app:                                                                                   
                                                                                                    
invalid Cosmos SDK mod path: [/home/clockwork/go/pkg/mod/github.com/cosmos/[email protected]/proto]

Guessing we need to explicitly supply the proto dir? or the default is not picked up correctly in ignite g commands?

Oh, I got it. It's a bug from another PR. I will fix it. Can you test using the chain serve command?

Pantani avatar Apr 23 '24 11:04 Pantani

Tried testing off this branch: Chain scaffolds ok.. On ignite g ts-client, I get this prompt:

Now ignite supports the `buf.build` (https://buf.build) registry to manage the protobuf dependencies. The embed protoc binary was deprecated and, your blockchain is still using it. Would you like to upgrade and add the `buf.build` config files to `proto/` folder: y/N

Answering No:

✘ Code generation using protoc is only supported by Ignite CLI v0.26.1 or older

Answering Yes:

 buf.build files added: 

/media/clockwork/Projects/Tendermint/venus//buf.gen.ts.yaml
/media/clockwork/Projects/Tendermint/venus//buf.lock
/media/clockwork/Projects/Tendermint/venus//buf.yaml
/media/clockwork/Projects/Tendermint/venus//buf.gen.gogo.yaml
/media/clockwork/Projects/Tendermint/venus//buf.gen.pulsar.yaml
/media/clockwork/Projects/Tendermint/venus//buf.gen.sta.yaml
/media/clockwork/Projects/Tendermint/venus//buf.gen.swagger.yaml

✗ Ignite proto directory path from the chain config doesn't match the proto directory path from the chain `buf.work.yaml`. Do you want to add the proto path `` to the directories list from the buf work file Y/N:

Answering Yes:

✘ Cannot build app:                                                                                                                                                                                                                                                                                                                                                                                                                               
                                                                                                                                                                                                                                                                                                                                                                                                                                                
error while running command /home/clockwork/go/bin/buf export /media/clockwork/Projects/Tendermint/venus/proto --output=/tmp/includeFolder1117887864: Failure: directory "." listed in /media/clockwork/Projects/Tendermint/venus/buf.work.yaml, it is not valid to have "." as a workspace directory, as this is no different than not having a workspace at all, see https://buf.build/docs/reference/workspaces/#directories for more details
: exit status 1

Answering No:

✘ Cannot build app:                                                                                   
                                                                                                    
invalid Cosmos SDK mod path: [/home/clockwork/go/pkg/mod/github.com/cosmos/[email protected]/proto]

Guessing we need to explicitly supply the proto dir? or the default is not picked up correctly in ignite g commands?

Oh, I got it. It's a bug from another PR. I will fix it. Can you test using the chain serve command?

not really...the whole idea is to see if the ignite g ts-client command works like before

clockworkgr avatar Apr 24 '24 15:04 clockworkgr

still unable to test this properly to check correctness of ts-client output

clockworkgr avatar Apr 26 '24 10:04 clockworkgr

still unable to test this properly to check correctness of ts-client output

sorry, you are right. The fix is already in the main branch and updated here. But I got another issue now, let me fix first

Pantani avatar Apr 26 '24 19:04 Pantani

We may need to throttle or cache BSR requests

✘ Cannot build app:                                                                                                                                                                                                                                                                                                                                                               
                                                                                                                                                                                                                                                                                                                                                                                
error while running command /opt/homebrew/bin/buf generate /Users/clockwork/Documents/Tendermint/venus/proto/venus/venus/query.proto --template=/Users/clockwork/Documents/Tendermint/venus/proto/buf.gen.ts.yaml --output=/Users/clockwork/Documents/Tendermint/venus/ts-client/cosmos.slashing.v1beta1/types --error-format=json --log-format=json: Failure: too many requests
                                                                                                                                                                                                                                                                                                                                                                                
Please see https://buf.build/docs/bsr/rate-limits for details about BSR rate limiting.                                                                                                                                                                                                                                                                                          
: exit status 1  

It exits early but the generated output up to that point looks good ...However it seems that the distinction between queries and Txs is not respected hence the codegen creates Tx methods for module queries...

clockworkgr avatar Apr 29 '24 07:04 clockworkgr

We may need to throttle or cache BSR requests

✘ Cannot build app:                                                                                                                                                                                                                                                                                                                                                               
                                                                                                                                                                                                                                                                                                                                                                                
error while running command /opt/homebrew/bin/buf generate /Users/clockwork/Documents/Tendermint/venus/proto/venus/venus/query.proto --template=/Users/clockwork/Documents/Tendermint/venus/proto/buf.gen.ts.yaml --output=/Users/clockwork/Documents/Tendermint/venus/ts-client/cosmos.slashing.v1beta1/types --error-format=json --log-format=json: Failure: too many requests
                                                                                                                                                                                                                                                                                                                                                                                
Please see https://buf.build/docs/bsr/rate-limits for details about BSR rate limiting.                                                                                                                                                                                                                                                                                          
: exit status 1  

It exits early but the generated output up to that point looks good ...However it seems that the distinction between queries and Txs is not respected hence the codegen creates Tx methods for module queries...

The problem now is the rate limit, we are calling the buf.build to often, I need to improve it

Pantani avatar Apr 29 '24 17:04 Pantani

I was just downloading.... the nearly 1gb main branch here.

This PR is great!

faddat avatar May 19 '24 16:05 faddat

This PR also speeds up the scaffold and sever commands and the CI test

Pantani avatar May 23 '24 18:05 Pantani