status-go change ipfs gateway and add Mode build param

trafficstars

The Infura IPFS gateway was deprecated so all stickers are broken.

here is a hotfix , that should be temporary, where I replace the gateway with the cloudflare one for now.

I tested it in Desktop, it works, but it's super slow and also has a limit, so it's very unreliable.

Aug 10 '22 19:08 jrainville

Pull Request Checklist

[ ] Have you updated the documentation, if impacted (e.g. docs.status.im)?
[ ] Have you tested changes with mobile?
[ ] Have you tested changes with desktop?

Aug 10 '22 19:08 status-github-bot[bot]

Jenkins Builds

Click to see older builds (42)

:grey_question:	Commit	:hash:	Finished (UTC)	Duration	Platform	Result
:heavy_check_mark:	e54e0961	#1	2022-08-10 19:35:06	~3 min	`linux`	:package:`zip`
:heavy_check_mark:	e54e0961	#1	2022-08-10 19:36:40	~5 min	`ios`	:package:`zip`
:heavy_check_mark:	e54e0961	#1	2022-08-10 19:37:12	~6 min	`android`	:package:`aar`

:heavy_check_mark:	bf7be38d	#2	2022-08-11 14:30:40	~2 min	`android`	:package:`aar`
:heavy_check_mark:	bf7be38d	#2	2022-08-11 14:30:55	~3 min	`ios`	:package:`zip`
:heavy_check_mark:	bf7be38d	#2	2022-08-11 14:31:53	~4 min	`linux`	:package:`zip`

:heavy_check_mark:	96655c23	#3	2022-08-11 15:28:02	~2 min	`ios`	:package:`zip`
:heavy_check_mark:	96655c23	#3	2022-08-11 15:29:27	~3 min	`android`	:package:`aar`
:heavy_check_mark:	96655c23	#3	2022-08-11 15:38:43	~13 min	`linux`	:package:`zip`

:heavy_check_mark:	3147cd09	#4	2022-08-16 15:40:51	~2 min	`ios`	:package:`zip`
:heavy_check_mark:	3147cd09	#4	2022-08-16 15:43:11	~5 min	`android`	:package:`aar`
:heavy_check_mark:	3147cd09	#4	2022-08-16 15:45:00	~6 min	`linux`	:package:`zip`

:heavy_check_mark:	00c335c5	#5	2022-08-19 19:46:32	~2 min	`ios`	:package:`zip`
:heavy_check_mark:	00c335c5	#5	2022-08-19 19:47:18	~2 min	`linux`	:package:`zip`
:heavy_check_mark:	00c335c5	#5	2022-08-19 19:48:54	~4 min	`android`	:package:`aar`

:heavy_check_mark:	6b45dc68	#6	2022-08-22 17:31:48	~2 min	`linux`	:package:`zip`
:heavy_check_mark:	6b45dc68	#6	2022-08-22 17:32:39	~3 min	`ios`	:package:`zip`
:heavy_check_mark:	6b45dc68	#6	2022-08-22 17:34:19	~5 min	`android`	:package:`aar`

:heavy_check_mark:	e6b126d1	#7	2022-08-22 21:27:47	~1 min	`linux`	:package:`zip`
:heavy_check_mark:	e6b126d1	#7	2022-08-22 21:29:20	~3 min	`android`	:package:`aar`
:heavy_check_mark:	e6b126d1	#7	2022-08-22 21:29:24	~3 min	`ios`	:package:`zip`

:heavy_check_mark:	c7ac6e60	#8	2022-08-23 17:55:11	~3 min	`linux`	:package:`zip`
:heavy_check_mark:	c7ac6e60	#8	2022-08-23 17:58:16	~6 min	`ios`	:package:`zip`
:heavy_check_mark:	c7ac6e60	#8	2022-08-23 17:59:44	~8 min	`android`	:package:`aar`

:heavy_check_mark:	e8da61d5	#9	2022-08-24 20:17:29	~2 min	`linux`	:package:`zip`
:heavy_check_mark:	e8da61d5	#9	2022-08-24 20:17:42	~2 min	`ios`	:package:`zip`
:heavy_check_mark:	e8da61d5	#9	2022-08-24 20:18:49	~3 min	`android`	:package:`aar`

:heavy_check_mark:	f78ad5ea	#10	2022-08-26 15:26:24	~1 min	`linux`	:package:`zip`
:heavy_check_mark:	f78ad5ea	#10	2022-08-26 15:27:20	~2 min	`ios`	:package:`zip`
:heavy_check_mark:	f78ad5ea	#10	2022-08-26 15:27:47	~3 min	`android`	:package:`aar`

:x:	b0046332	#11	2022-08-26 15:30:42	~31 sec	`ios`	:page_facing_up:`log`
:x:	b0046332	#11	2022-08-26 15:30:59	~52 sec	`android`	:page_facing_up:`log`
:x:	b0046332	#11	2022-08-26 15:31:37	~1 min	`linux`	:page_facing_up:`log`

:heavy_check_mark:	982c5b3e	#12	2022-08-26 15:37:40	~2 min	`ios`	:package:`zip`
:heavy_check_mark:	982c5b3e	#12	2022-08-26 15:42:44	~7 min	`linux`	:package:`zip`
:heavy_check_mark:	982c5b3e	#12	2022-08-26 15:43:35	~8 min	`android`	:package:`aar`

:heavy_check_mark:	bbfa5e9e	#13	2022-08-26 15:52:00	~2 min	`ios`	:package:`zip`
:heavy_check_mark:	bbfa5e9e	#13	2022-08-26 15:56:29	~6 min	`linux`	:package:`zip`
:heavy_check_mark:	bbfa5e9e	#13	2022-08-26 15:58:10	~8 min	`android`	:package:`aar`

:heavy_check_mark:	919b9aa1	#14	2022-08-26 16:35:14	~2 min	`ios`	:package:`zip`
:heavy_check_mark:	919b9aa1	#14	2022-08-26 16:36:51	~3 min	`linux`	:package:`zip`
:heavy_check_mark:	919b9aa1	#14	2022-08-26 16:37:35	~4 min	`android`	:package:`aar`

:grey_question:	Commit	:hash:	Finished (UTC)	Duration	Platform	Result
:heavy_check_mark:	0152e10d	#15	2022-08-26 16:48:40	~1 min	`linux`	:package:`zip`
:heavy_check_mark:	0152e10d	#15	2022-08-26 16:49:16	~2 min	`ios`	:package:`zip`
:heavy_check_mark:	0152e10d	#15	2022-08-26 16:50:11	~3 min	`android`	:package:`aar`

:heavy_check_mark:	86054875	#16	2022-08-26 16:59:51	~1 min	`linux`	:package:`zip`
:heavy_check_mark:	86054875	#16	2022-08-26 17:01:04	~2 min	`ios`	:package:`zip`
:heavy_check_mark:	86054875	#16	2022-08-26 17:01:45	~3 min	`android`	:package:`aar`

Aug 10 '22 19:08 status-im-auto

hey @jrainville @richard-ramos i've created new IPFS project on infura, so now we have a dedicated subdomain, we should use it, but first we need to move pins i guess https://blog.infura.io/post/migrate-your-files-to-infuras-new-ipfs-service-in-3-easy-steps#how-to-pin-ipfs-data-already-hosted-by-infura

Aug 11 '22 11:08 flexsurfer

probably we should use it only for production and keep cloudflare for develop

Aug 11 '22 11:08 flexsurfer

Thanks for the reviews and test. Sadly for you, you'll have to do it again. I improved this PR significantly.

I added the mode build param. That means that we can now pass if the builds DEV or PROD (or more if we want).

I coded the passing of that param in desktop, sending it from mobile should be similar.

Anyway, when we are in dev, we use the status gateway, in prod we use the infura one.

I default to dev when mode is undefined, let me know if you want the opposite.

Aug 19 '22 19:08 jrainville

@jrainville as we need it on status-react side for release, can you please also add VERSION bump in your PR? Thanks!

Aug 22 '22 10:08 churik

Adding weird mode type arguments is just confusing. We should be instead setting this like we set the OpenSea API URL: https://github.com/status-im/status-go/blob/fea37ff5b46ff84153591d327f4672b7d3c45915/params/config.go#L511-L514 Which is part of the NodeConfig: https://github.com/status-im/status-go/blob/fea37ff5b46ff84153591d327f4672b7d3c45915/params/config.go#L454-L455 Which in mobile we normally pass via RPC when creating accounts or when changing settings:

(def login-node-config
  {:WalletConfig (cond-> {:Enabled true}
                   (not= config/opensea-api-key "")
                   (assoc :OpenseaAPIKey config/opensea-api-key))})

https://github.com/status-im/status-mobile/blob/8ec2f23203dd1643205a8ff022ff4e4530b7e905/src/status_im/node/core.cljs#L63-L66 Which is passed to the node via the Statusgo.saveAccountAndLogin call in saveAccountAndLogin:

    public void saveAccountAndLogin(final String multiaccountData, final String password, final String settings, final String config, final String accountsData) {
        try {
            Log.d(TAG, "saveAccountAndLogin");
            String finalConfig = prepareDirAndUpdateConfig(config, this.getKeyUID(multiaccountData));
            String result = Statusgo.saveAccountAndLogin(multiaccountData, password, settings, finalConfig, accountsData);

https://github.com/status-im/status-mobile/blob/526835fcf062acc35b2582a9c69af10d9ce3a67e/modules/react-native-status/android/src/main/java/im/status/ethereum/module/StatusModule.java#L323-L327

This way the setting is not passed at build time, which generates more build variations, but also allows for more easily changing such a setting by the user, if they want to specify they own URL.

Aug 22 '22 10:08 jakubgs

There is also LoginWithConfig function which can pass the NodeConfig for an already created account:

    let response = status_go.loginWithConfig($payload, hashedPassword, nodeCfgObj)

https://github.com/status-im/status-desktop/blob/70a1381e8606b975c829c9e63382033a23dd45ac/src/backend/accounts.nim#L254

Aug 22 '22 10:08 jakubgs

Are we sure this is how we want to do it? Wouldn't it make more sense to do it right the first time and put it in the Node config so it can be changed via RPC by user?

I think unless we allow the user to actually set it in this PR (and in the UI), it would be safer not to store it in the database, since that would be sort of a field that we would not really use, but we'd still have to migrate or it might be susceptible to changes Essentially we don't have to commit for now, while creating a migration would be fairly committal, it's easy to move from this approach to storing to the db, while it's a bit more complicated to say move from storing to not storing. But I am honestly fine with any approach.

Aug 22 '22 10:08 cammellos

We do the same thing for Infura RPC URL: https://github.com/status-im/status-go/blob/ad326fa290c73a6ca5c9c07ddf607b8007111f12/params/config.go#L503-L506 It's also part of config.

Aug 22 '22 10:08 jakubgs

I understand how migrations are a pain, and that this is more of a hotfix, so I'm not saying we HAVE TO use the config.

But you should be aware that we'll have to use it EVENTUALLY if we want users to be able to change this setting.

EDIT: And I agree with @cammellos that Mode is a bad name, far too generic.

Aug 22 '22 10:08 jakubgs

I agree that Mode is a bad name.

So, I think the consensus for now is to just rename the params to something better like suggested and then, if needed, we can move to a node config approach?

Aug 22 '22 16:08 jrainville

sounds good

On Mon, 22 Aug 2022, 17:52 Jonathan Rainville, @.***> wrote:

I agree that Mode is a bad name.

So, I think the consensus for now is to just rename the params to something better like suggested and then, if needed, we can move to a node config approach?

— Reply to this email directly, view it on GitHub https://github.com/status-im/status-go/pull/2795#issuecomment-1222631623, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHYJMBOSODAWIYMHQL2QF3V2OV4PANCNFSM56FV7OZA . You are receiving this because you were mentioned.Message ID: @.***>

Aug 22 '22 16:08 cammellos

Update done

Aug 22 '22 17:08 jrainville

hey @jrainville for some reason stickers_market doesn't return anything in mobile with this PR

Aug 24 '22 12:08 flexsurfer

oh thats because i need to add parameter in mobile makefile ?

Aug 24 '22 12:08 flexsurfer

@flexsurfer with the latest build, it should default to the status Gateway if none is provided thanks to this line https://github.com/status-im/status-go/pull/2795/files#diff-76ed074a9305c04054cdebb9e9aad2d818052b07091de1f20cad0bbac34ffb52R10

I'm not sure why it didn't work for you. I can test again later to see if the default value works.

Otherwise, passing the param from mobile would also work.

Aug 24 '22 14:08 jrainville

@flexsurfer I just tested by removing the param from the Status-desktop Makefile and the fault was set correctly for me.

Make sure status-go rebuilds correctly and you use the right commit maybe?

Also, the loading is a bit slower with the status gateway (dev) than the infura one, so maybe you just need to wait a little?

Aug 24 '22 20:08 jrainville

hey @jrainville the problem is not in loading, the problem is that stickers_market returns an empty value

Aug 25 '22 07:08 flexsurfer

just tested with makefile value, same, empty result

Aug 25 '22 08:08 flexsurfer

just wondering why there is no error but empty result

Aug 25 '22 10:08 flexsurfer

stickers_market response {"jsonrpc":"2.0","id":1,"result":null}

Aug 25 '22 11:08 flexsurfer

Very weird. I thought it was the IPFS gateway that was causing trouble, but this is something else. Is this happening with mainnet? or is it in a different chain?

Aug 25 '22 12:08 richard-ramos

mainnet

Aug 25 '22 12:08 flexsurfer

btw i used this PR for testing https://github.com/status-im/status-mobile/pull/13850

Aug 25 '22 12:08 flexsurfer

can it be something with hashes ?

Aug 25 '22 12:08 flexsurfer

can it be something with hashes ?

It,s possible. The new commit hash is e8da61d5566397ed8bd574eef52823f6df2c1846. I amend the previous commit each time I do a change or rebase, that's why the hash changes each time.

Aug 25 '22 14:08 jrainville

i mean ipfs hashes

Aug 25 '22 18:08 flexsurfer

I don't think so. It works perfectly on Desktop. I even created an e2e test for it

Aug 25 '22 18:08 jrainville

t=2022-08-26T08:42:46+0000 lvl=warn msg="Could not retrieve stickerpack data" packID=3 error="Get \"QmbYzUKEPM3X2Yi7dXZVbNcbUz9nMJ4M6DNtpCC5kKbjh6\": unsupported protocol scheme \"\""
t=2022-08-26T08:42:47+0000 lvl=warn msg="Could not retrieve stickerpack data" packID=4 error="Get \"Qmcehb2dboemjLXDqLZ2V6Vf8w1Z1MVje9QSErERCMMUYs\": unsupported protocol scheme \"\""
t=2022-08-26T08:42:47+0000 lvl=warn msg="Could not retrieve stickerpack data" packID=5 error="Get \"QmeR9LuF8b4QG5HpsmN3gJCXbFiWy5Zki6DDYdcNSuqawn\": unsupported protocol scheme \"\""
t=2022-08-26T08:42:47+0000 lvl=warn msg="Could not retrieve stickerpack data" packID=6 error="Get \"QmWWB4D8e5iGszLe4peGuNf6pqMef4hr233EKRHq9pnbQu\": unsupported protocol scheme \"\""
t=2022-08-26T08:42:48+0000 lvl=warn msg="Could not retrieve stickerpack data" packID=7 error="Get \"QmdMRLZzE6hkdUWCZkULEpzNi5hhKQXp4fk4zxiEHV1dPz\": unsupported protocol scheme \"\""
t=2022-08-26T08:42:48+0000 lvl=warn msg="Could not retrieve stickerpack data" packID=8 error="Get \"QmWAhePgrMqKcmBMBX6rVtA1LiWQ8YgJcQ7WHbB2N4u6HD\": unsupported protocol scheme \"\""
t=2022-08-26T08:42:48+0000 lvl=warn msg="Could not retrieve stickerpack data" packID=9 error="Get \"QmUB1dD3syoU6pgF92CvvRDSgpBuS5JFKFxjFMrqAQmYLz\": unsupported protocol scheme \"\""
t=2022-08-26T08:42:49+0000 lvl=warn msg="Could not retrieve stickerpack data" packID=10 error="Get \"QmPg9DD1JaPDm5mi8XG8AAYTYtuxSseAvESQZr45V66NPk\": unsupported protocol scheme \"\""
t=2022-08-26T08:42:49+0000 lvl=warn msg="Could not retrieve stickerpack data" packID=11 error="Get \"QmRXF9LKeE7vEJhxpSRCMf9CDv7UdB1saczNTfQXYQTqU7\": unsupported protocol scheme \"\""
t=2022-08-26T08:42:49+0000 lvl=warn msg="Could not retrieve stickerpack data" packID=12 error="Get \"QmPF2C8DZCw6QonaLC9iggYrrq3JwuxKdHwU6FFeeVqhVd\": unsupported protocol scheme \"\""
t=2022-08-26T08:42:50+0000 lvl=warn msg="Could not retrieve stickerpack data" packID=13 error="Get \"QmWjtD7YjTVBM2WKSoZB6e6R8M2KMHamQy2em9Sa65jB1E\": unsupported protocol scheme \"\""
t=2022-08-26T08:42:50+0000 lvl=warn msg="Could not retrieve stickerpack data" packID=14 error="Get \"QmUKQ4HM5Qu7v1hoQSY9Q9nAPBBTZNnAFWvNvhxfaTNmvo\": unsupported protocol scheme \"\""
t=2022-08-26T08:42:50+0000 lvl=warn msg="Could not retrieve stickerpack data" packID=15 error="Get \"QmSbA2as2ShK3V5cLLj6ujrUjsy9zwSmyQhocXNab3DRQP\": unsupported protocol scheme \"\""
t=2022-08-26T08:42:50+0000 lvl=info msg="Messenger failed to process sent messages" error="Can't get raw message with id 0x1a9b1bd726530f8ede066da56a7746e442c96e78234472922de8693eed2bfe58: sql: no rows in result set"
t=2022-08-26T08:42:51+0000 lvl=warn msg="Could not retrieve stickerpack data" packID=16 error="Get \"QmZ6geozxVJR5iTBW5Uqm3UcUK7GVkcKBfoy7LvSYqSjSF\": unsupported protocol scheme \"\""
t=2022-08-26T08:42:51+0000 lvl=warn msg="STICKER PACKS :: DONE fetching "
t=2022-08-26T08:42:51+0000 lvl=warn msg="Could not retrieve stickerpack data" packID=17 error="Get \"QmZGFJBpby5QFe5Ga7e4G7nPKDkGsCTvkz7oEhoKWnPp3L\": unsupported protocol scheme \"\""
t=2022-08-26T08:42:51+0000 lvl=warn msg="Could not retrieve stickerpack data" packID=18 error="Get \"QmWP9WD7jaVxncNQp7LJfQmPzCpeAgVRV2rAKkd3M6Xh9Z\": unsupported protocol scheme \"\""
t=2022-08-26T08:42:52+0000 lvl=warn msg="Could not retrieve stickerpack data" packID=19 error="Get \"QmfEQGLCz7UJqhJ2wxqDMjc7FePAGPBJrwqZ3WYKTmww49\": unsupported protocol scheme \"\""
t=2022-08-26T08:42:52+0000 lvl=warn msg="STICKER PACKS :: PACK - " LOG15_ERROR= LOG15_ERROR="Normalized odd number of arguments by adding nil"
t=2022-08-26T08:42:52+0000 lvl=warn msg="STICKER PACKS :: PACK - " LOG15_ERROR= LOG15_ERROR="Normalized odd number of arguments by adding nil"
t=2022-08-26T08:42:52+0000 lvl=warn msg="STICKER PACKS :: PACK - " LOG15_ERROR= LOG15_ERROR="Normalized odd number of arguments by adding nil"
t=2022-08-26T08:42:52+0000 lvl=warn msg="STICKER PACKS :: ERR " LOG15_ERROR= LOG15_ERROR="Normalized odd number of arguments by adding nil"
t=2022-08-26T08:42:52+0000 lvl=warn msg="STICKER PACKS :: ERR " LOG15_ERROR= LOG15_ERROR="Normalized odd number of arguments by adding nil"
t=2022-08-26T08:42:52+0000 lvl=warn msg="STICKER PACKS :: " LOG15_ERROR= LOG15_ERROR="Normalized odd number of arguments by adding nil"

Aug 26 '22 08:08 flexsurfer

status-go status-go copied to clipboard

change ipfs gateway and add Mode build param

Pull Request Checklist

Jenkins Builds

status-go
status-go copied to clipboard