cosmospy icon indicating copy to clipboard operation
cosmospy copied to clipboard

REST endpoint migration

Open shivlim opened this issue 4 years ago • 38 comments

Hi, I see REST endpoint /txs has been migrated now and replaced by /cosmos/tx/v1beta1/txs And that needs enoded transaction string using protobuf. Any plans to support this?

https://docs.cosmos.network/master/migrations/rest.html

Thanks

shivlim avatar Nov 21 '21 01:11 shivlim

A PR is welcome! I may not have the time to do the migration in time myself.

hukkin avatar Nov 21 '21 08:11 hukkin

Does anyone have a hacky fix for this? This issue has broke the discord faucet bot I am maintaining and would like to bring it back up someone. Any help would be much appreciated!

czarcas7ic avatar Dec 13 '21 17:12 czarcas7ic

It's not a small change imo. it hasn't been migrated to /cosmos/tx/v1beta1/txs. It has more been removed. To quote the cosmos docs:

It is not possible to generate or sign a transaction using REST, only to broadcast one. In order to broadcast a transaction using REST, you will need to generate, sign, and encode the transaction using either the CLI or programmatically with Go.

I feel we will need some bigger changes here.

Broadcasting a transaction using the REST endpoint (served by gRPC-gateway) can be done by sending a POST request as follows, where the txBytes are the protobuf-encoded bytes of a signed transaction:

ctrl-Felix avatar Dec 17 '21 22:12 ctrl-Felix

Transaction generation and signing is done fully offline by cosmospy already, only broadcasting happens via REST endpoint. So IIUC I don't think the difference is that huge. The new endpoint just probably wants the transaction data in a different format. (I imagine, I still haven't looked into this.)

hukkin avatar Dec 17 '21 22:12 hukkin

Transaction generation and signing is done fully offline by cosmospy already, only broadcasting happens via REST endpoint. So IIUC I don't think the difference is that huge. The new endpoint just probably wants the transaction data in a different format. (I imagine, I still haven't looked into this.)

No you can't submit it by the rest api anymore. But I think we just can change to the rpc endpoint which is /broadcast_tx_async

The rest stays the same

ctrl-Felix avatar Dec 17 '21 22:12 ctrl-Felix

No you can't submit it by the rest api anymore.

What makes you think that? There's a mapping from legacy to new REST endpoints here https://docs.cosmos.network/master/migrations/rest.html#migrating-to-new-rest-endpoints. Why would it not be useful?

hukkin avatar Dec 17 '21 23:12 hukkin

@ctrl-Felix I think you are right, but before it gets passed doesn't it have to manually be encoded with protobuf?

czarcas7ic avatar Dec 17 '21 23:12 czarcas7ic

I tried it @hukkin it's not working

ctrl-Felix avatar Dec 17 '21 23:12 ctrl-Felix

Please check #27 Please try it on your setup before

ctrl-Felix avatar Dec 17 '21 23:12 ctrl-Felix

I tried it @hukkin it's not working

Yeah because the new endpoint wants the transaction in a different format. It won't work without changes to cosmospy. That does not mean that the new endpoint is not working or can not be used.

hukkin avatar Dec 17 '21 23:12 hukkin

I didn't say that it can't be used and I figured out my first message might be wrong. What I actually tried is the rpc endpoint and it's definetly working

ctrl-Felix avatar Dec 17 '21 23:12 ctrl-Felix

@ctrl-Felix You said

In fact it's post and working the same way /txs is

If that is true I will be so mad

And also thankful for you teaching me something haha

czarcas7ic avatar Dec 17 '21 23:12 czarcas7ic

I tried it on chihuahua because I needed it there. I will try it on other chains now

ctrl-Felix avatar Dec 17 '21 23:12 ctrl-Felix

Forget what I said. I messed something up. Lol Sorry guys, I have to work out some things

ctrl-Felix avatar Dec 17 '21 23:12 ctrl-Felix

No problem, please keep us updated if you can though. This issue has completely shut down our faucet which is not ideal, appreciate the work you are doing!

czarcas7ic avatar Dec 17 '21 23:12 czarcas7ic

@hukkin I don't know if they didn't update swagger ot if that's the new endpoint. But check out the docs at node.atomscan.com

This endpoint requires following data: { "tx_bytes": "string", "mode": "BROADCAST_MODE_UNSPECIFIED" }

ctrl-Felix avatar Dec 17 '21 23:12 ctrl-Felix

Yeah that's correct. Here's the official swagger btw https://v1.cosmos.network/rpc

We still need to know how "tx_bytes" is encoded and what the possible options for "mode" are.

hukkin avatar Dec 17 '21 23:12 hukkin

@hukkin there is:

BROADCAST_MODE_UNSPECIFIED BROADCAST_MODE_BLOCK BROADCAST_MODE_SYNC BROADCAST_MODE_ASYNC

And tx_bytes is encoded with gogoprotobuf

https://docs.cosmos.network/master/core/encoding.html

czarcas7ic avatar Dec 17 '21 23:12 czarcas7ic

I tried hard to get protobuf bytes but couldnt in python. So i ended up using cosmjs

shivlim avatar Dec 17 '21 23:12 shivlim

We're gonna need to depend on and use https://github.com/protocolbuffers/protobuf I expect.

This also seems relevant https://docs.cosmos.network/master/architecture/adr-020-protobuf-transaction-encoding.html

hukkin avatar Dec 17 '21 23:12 hukkin

Protobuf docs for python are a pain...

Everything's focussed on go

ctrl-Felix avatar Dec 18 '21 13:12 ctrl-Felix

I think I've a fixed version now. I found the cyberai fork which already implemented all protibufs (https://github.com/SaveTheAles/cyberpy). I basically took that one and adapted it to accept custom coin denoms and prefixes.

ctrl-Felix avatar Dec 24 '21 12:12 ctrl-Felix

#28 That's the changes I would propose. Feel free to try it out

ctrl-Felix avatar Dec 24 '21 12:12 ctrl-Felix

anyone got any solution for this problem ?? tnx

meysamkheyrollah avatar Jan 02 '22 23:01 meysamkheyrollah

@meysamkheyrollah ctrl-Felix fix worked for me but had to slightly modify it for what I was using it for.

czarcas7ic avatar Jan 03 '22 00:01 czarcas7ic

@czarcas7ic tnx

meysamkheyrollah avatar Jan 03 '22 20:01 meysamkheyrollah

@czarcas7ic is ctrl-Felix solution works in general for sending transactions ??

meysamkheyrollah avatar Jan 03 '22 20:01 meysamkheyrollah

@meysamkheyrollah ctrl-Felix fix worked for me but had to slightly modify it for what I was using it for.

What did you change? Did you use the rpc endpoint?

ctrl-Felix avatar Jan 05 '22 10:01 ctrl-Felix

I believe this is what they changed https://github.com/hukkin/cosmospy/pull/28#issuecomment-1001075689

I think we should make cosmospy agnostic of the endpoint used, and only return the tx_b64 transaction data that both of the endpoints share. And then document the use of both endpoints in README.md using the httpx library.

hukkin avatar Jan 05 '22 11:01 hukkin

I believe this is what they changed https://github.com/hukkin/cosmospy/pull/28#issuecomment-1001075689

I think we should make cosmospy agnostic of the endpoint used, and only return the tx_b64 transaction data that both of the endpoints share. And then document the use of both endpoints in README.md using the httpx library.

Definetly a good idea. I hope to be able to do some further research this week

ctrl-Felix avatar Jan 05 '22 13:01 ctrl-Felix

Please does anyone know how to convert the "tx json" to "tx_bytes" needed for the transaction broadcast request ? Is there any library ?

ZakariaTDF avatar Jan 10 '22 17:01 ZakariaTDF

Hey,

I have used the PR #28 that @ctrl-Felix provided (btw Thank you!) and it is working for me.

As you can see in https://github.com/ctrl-Felix/cosmospy/blob/f21794bf547264daf07e72b3b7b0ee12f86b70f2/src/cosmospy/_transaction.py#L28 you have to use the library and provide all the details as in init.

Then you have to "add_transfer" and after that use "get_pushable" where you will receive this response https://github.com/ctrl-Felix/cosmospy/blob/f21794bf547264daf07e72b3b7b0ee12f86b70f2/src/cosmospy/_transaction.py#L77 , tx_b64 is your tx in bytes. Then you simply send POST with such data: {"tx_bytes": tx_b64, "mode": "BROADCAST_MODE_SYNC"}

mangekyousharingan avatar Jan 10 '22 17:01 mangekyousharingan

@mangekyousharingan thanks for your answer , i can generate the tx_64 but the transaction doesn't succeed when i send the request even though it's a 200 status code, do you have any idea about the problem could be ? (i double checked the seed phrase) . This is the response i get from the server :

{ "tx_response": { "height": "0", "txhash": "EEF1568353B5BAA5C0E6E3D41D5D9A75C399322BA4E6F92EA525B6355D8167DF", "codespace": "sdk", "code": 19, "data": "", "raw_log": "", "logs": [], "info": "", "gas_wanted": "0", "gas_used": "0", "tx": null, "timestamp": "" } }

ZakariaTDF avatar Jan 11 '22 09:01 ZakariaTDF

I'd appreciate it if you didnt't use this thread for support requests. I (and many others) receive an email every time you post.

Please consider going to a more appropriate forum, or at least creating a separate issue that I can unsubscribe.

hukkin avatar Jan 11 '22 09:01 hukkin

Hey,

As I said before @ctrl-Felix solution works for me and tx is being send, but however when I'm trying to send tx with memo, the memo is not included in tx and then not visible on blockchain. Is there a way to fix it or I am missing somehting?

Appreciate the work here!

mangekyousharingan avatar Jan 14 '22 14:01 mangekyousharingan

Hey,

As I said before @ctrl-Felix solution works for me and tx is being send, but however when I'm trying to send tx with memo, the memo is not included in tx and then not visible on blockchain. Is there a way to fix it or I am missing somehting?

Appreciate the work here!

I will add that to the updated version. The current pull request is just a dirty workaround to get it working

ctrl-Felix avatar Jan 16 '22 15:01 ctrl-Felix

Hey, As I said before @ctrl-Felix solution works for me and tx is being send, but however when I'm trying to send tx with memo, the memo is not included in tx and then not visible on blockchain. Is there a way to fix it or I am missing somehting? Appreciate the work here!

I will add that to the updated version. The current pull request is just a dirty workaround to get it working

I am interested in solving this issue quickly, please lead me to how can I help? @hukkin

Bha91 avatar Mar 08 '22 18:03 Bha91

@Bha91 This comment https://github.com/hukkin/cosmospy/issues/24#issuecomment-1005589846 and The PR https://github.com/hukkin/cosmospy/pull/28 and the review there should give you lots of pointers. Unfortunately I don't have more time to dedicate to this (unless funded), but will review your PR if you make one.

hukkin avatar Mar 12 '22 08:03 hukkin