status-mobile [#19232] - Fix derivation path generation

fixes #19232

Summary

When we create a new account, the derivation path should always be a new one unless we want to recover a previously created account.

We were generating derivation paths based on the number of existing accounts in the device, so that when a derived account, but the last, was removed, the derivation path suggested created the error reported in this PR's issue.

After verious dicussions with desktop devs we realized we needed to perform that generation in status-go.

This PR is using the endpoint "accounts_resolveSuggestedPathForKeypair" to get the correct derivation path.

Platforms

Android
iOS

Steps to test

Open Status
Create a new account
Go to wallet tab -> Add a new wallet account, the derivation path should be displayed and work fine.
Add many new wallet accounts as you wish
Remove some of them
Add new accounts

Everything should work now, and the issue reported shouldn't be reproducible.

status: ready

Apr 04 '24 22:04 ulisesmac

Jenkins Builds

Click to see older builds (67)

:grey_question:	Commit	:hash:	Finished (UTC)	Duration	Platform	Result
:heavy_check_mark:	d3472e1dbbc542aae4a0deff25c1a2d7c894d384	#1	2024-04-04 22:50:52	~8 min	`android`	:robot:`apk` :calling:
:heavy_check_mark:	d3472e1dbbc542aae4a0deff25c1a2d7c894d384	#1	2024-04-04 22:50:53	~8 min	`android-e2e`	:robot:`apk` :calling:
:heavy_check_mark:	d3472e1dbbc542aae4a0deff25c1a2d7c894d384	#1	2024-04-04 22:52:31	~10 min	`ios`	:iphone:`ipa` :calling:

:x:	d33b0dce20762214d83810ace14924f91ad1c2db	#2	2024-04-08 23:56:55	~2 min	`tests`	:page_facing_up:`log`
:heavy_check_mark:	d33b0dce20762214d83810ace14924f91ad1c2db	#2	2024-04-09 00:01:32	~7 min	`android-e2e`	:robot:`apk` :calling:
:heavy_check_mark:	d33b0dce20762214d83810ace14924f91ad1c2db	#2	2024-04-09 00:01:44	~7 min	`android`	:robot:`apk` :calling:
:heavy_check_mark:	d33b0dce20762214d83810ace14924f91ad1c2db	#2	2024-04-09 00:03:05	~8 min	`ios`	:iphone:`ipa` :calling:

:x:	e7185bff137db2848aa2652b6877f3c9b796a552	#3	2024-04-11 01:59:59	~2 min	`tests`	:page_facing_up:`log`
:heavy_check_mark:	e7185bff137db2848aa2652b6877f3c9b796a552	#3	2024-04-11 02:05:21	~7 min	`android-e2e`	:robot:`apk` :calling:
:heavy_check_mark:	e7185bff137db2848aa2652b6877f3c9b796a552	#3	2024-04-11 02:05:24	~8 min	`android`	:robot:`apk` :calling:
:heavy_check_mark:	e7185bff137db2848aa2652b6877f3c9b796a552	#3	2024-04-11 02:06:52	~9 min	`ios`	:iphone:`ipa` :calling:

:x:	8f51babd432649a7a4f87119f12a6d0565709504	#4	2024-04-16 15:26:35	~2 min	`tests`	:page_facing_up:`log`
:heavy_check_mark:	8f51babd432649a7a4f87119f12a6d0565709504	#4	2024-04-16 15:31:50	~7 min	`android-e2e`	:robot:`apk` :calling:
:heavy_check_mark:	8f51babd432649a7a4f87119f12a6d0565709504	#4	2024-04-16 15:32:00	~8 min	`android`	:robot:`apk` :calling:
:heavy_check_mark:	8f51babd432649a7a4f87119f12a6d0565709504	#4	2024-04-16 15:33:46	~9 min	`ios`	:iphone:`ipa` :calling:

:x:	6297318112ab9ccdad14d500067c2d8abf2b311c	#5	2024-04-19 09:41:23	~2 min	`tests`	:page_facing_up:`log`
:heavy_check_mark:	6297318112ab9ccdad14d500067c2d8abf2b311c	#5	2024-04-19 09:46:28	~7 min	`android-e2e`	:robot:`apk` :calling:
:heavy_check_mark:	6297318112ab9ccdad14d500067c2d8abf2b311c	#5	2024-04-19 09:46:34	~7 min	`android`	:robot:`apk` :calling:
:heavy_check_mark:	6297318112ab9ccdad14d500067c2d8abf2b311c	#5	2024-04-19 09:48:18	~9 min	`ios`	:iphone:`ipa` :calling:

:x:	5bb42e5830e145da65241e0c7f2521fc0717a0ec	#6	2024-04-19 14:31:35	~2 min	`tests`	:page_facing_up:`log`
:heavy_check_mark:	5bb42e5830e145da65241e0c7f2521fc0717a0ec	#6	2024-04-19 14:36:25	~7 min	`android-e2e`	:robot:`apk` :calling:
:heavy_check_mark:	5bb42e5830e145da65241e0c7f2521fc0717a0ec	#6	2024-04-19 14:36:33	~7 min	`android`	:robot:`apk` :calling:
:heavy_check_mark:	5bb42e5830e145da65241e0c7f2521fc0717a0ec	#6	2024-04-19 14:38:57	~9 min	`ios`	:iphone:`ipa` :calling:

:x:	1d5c0eed53ef4f9c90bef6b32e701e81b52ad969	#7	2024-05-17 16:21:23	~2 min	`tests`	:page_facing_up:`log`
:heavy_check_mark:	1d5c0eed53ef4f9c90bef6b32e701e81b52ad969	#7	2024-05-17 16:26:36	~7 min	`android-e2e`	:robot:`apk` :calling:
:heavy_check_mark:	1d5c0eed53ef4f9c90bef6b32e701e81b52ad969	#7	2024-05-17 16:26:41	~7 min	`android`	:robot:`apk` :calling:
:heavy_check_mark:	1d5c0eed53ef4f9c90bef6b32e701e81b52ad969	#7	2024-05-17 16:27:39	~8 min	`ios`	:iphone:`ipa` :calling:

:x:	93c3c1e20b0bf72ed7b88392524757564594f28f	#8	2024-05-17 22:22:18	~2 min	`tests`	:page_facing_up:`log`
:heavy_check_mark:	93c3c1e20b0bf72ed7b88392524757564594f28f	#8	2024-05-17 22:27:23	~8 min	`android-e2e`	:robot:`apk` :calling:
:heavy_check_mark:	93c3c1e20b0bf72ed7b88392524757564594f28f	#8	2024-05-17 22:27:55	~8 min	`ios`	:iphone:`ipa` :calling:
:heavy_check_mark:	93c3c1e20b0bf72ed7b88392524757564594f28f	#8	2024-05-17 22:28:16	~8 min	`android`	:robot:`apk` :calling:

:x:	d465cbdb47f223bee6f41f24eea7e8ff773d4450	#10	2024-05-20 15:30:14	~3 min	`tests`	:page_facing_up:`log`
:heavy_check_mark:	d465cbdb47f223bee6f41f24eea7e8ff773d4450	#10	2024-05-20 15:34:28	~7 min	`android-e2e`	:robot:`apk` :calling:
:heavy_check_mark:	d465cbdb47f223bee6f41f24eea7e8ff773d4450	#10	2024-05-20 15:34:33	~7 min	`android`	:robot:`apk` :calling:
:heavy_check_mark:	d465cbdb47f223bee6f41f24eea7e8ff773d4450	#10	2024-05-20 15:37:42	~10 min	`ios`	:iphone:`ipa` :calling:

:x:	693d1756b09d1a81c4fcbbb15241989b1670afdf	#11	2024-05-20 19:18:52	~2 min	`tests`	:page_facing_up:`log`
:heavy_check_mark:	693d1756b09d1a81c4fcbbb15241989b1670afdf	#11	2024-05-20 19:23:27	~7 min	`android-e2e`	:robot:`apk` :calling:
:heavy_check_mark:	693d1756b09d1a81c4fcbbb15241989b1670afdf	#11	2024-05-20 19:23:29	~7 min	`android`	:robot:`apk` :calling:
:heavy_check_mark:	693d1756b09d1a81c4fcbbb15241989b1670afdf	#11	2024-05-20 19:25:02	~8 min	`ios`	:iphone:`ipa` :calling:

:x:	e442a1c97d1e6849a222e69feb83e2744eb30025	#12	2024-05-20 19:45:12	~2 min	`tests`	:page_facing_up:`log`
:heavy_check_mark:	e442a1c97d1e6849a222e69feb83e2744eb30025	#12	2024-05-20 19:50:16	~7 min	`android-e2e`	:robot:`apk` :calling:
:heavy_check_mark:	e442a1c97d1e6849a222e69feb83e2744eb30025	#12	2024-05-20 19:50:24	~7 min	`android`	:robot:`apk` :calling:
:heavy_check_mark:	e442a1c97d1e6849a222e69feb83e2744eb30025	#12	2024-05-20 19:51:06	~8 min	`ios`	:iphone:`ipa` :calling:

:x:	c78bd68da3a487db2d07281e374295a4ca78ee18	#13	2024-05-20 22:26:47	~2 min	`tests`	:page_facing_up:`log`
:heavy_check_mark:	c78bd68da3a487db2d07281e374295a4ca78ee18	#13	2024-05-20 22:30:15	~5 min	`android`	:robot:`apk` :calling:
:heavy_check_mark:	c78bd68da3a487db2d07281e374295a4ca78ee18	#13	2024-05-20 22:31:09	~6 min	`android-e2e`	:robot:`apk` :calling:
:heavy_check_mark:	c78bd68da3a487db2d07281e374295a4ca78ee18	#13	2024-05-20 22:33:12	~8 min	`ios`	:iphone:`ipa` :calling:

:x:	649a2e2a47287555ef6678a9f19e857a9f03aa83	#14	2024-05-21 16:12:29	~2 min	`tests`	:page_facing_up:`log`
:heavy_check_mark:	649a2e2a47287555ef6678a9f19e857a9f03aa83	#14	2024-05-21 16:17:45	~7 min	`android-e2e`	:robot:`apk` :calling:
:heavy_check_mark:	649a2e2a47287555ef6678a9f19e857a9f03aa83	#14	2024-05-21 16:17:55	~8 min	`android`	:robot:`apk` :calling:
:heavy_check_mark:	649a2e2a47287555ef6678a9f19e857a9f03aa83	#14	2024-05-21 16:18:32	~8 min	`ios`	:iphone:`ipa` :calling:

:x:	5bbc9d5a88bd906da62c96ae10c3ac43c64f6dda	#15	2024-05-21 16:53:04	~2 min	`tests`	:page_facing_up:`log`
:heavy_check_mark:	5bbc9d5a88bd906da62c96ae10c3ac43c64f6dda	#15	2024-05-21 16:58:33	~8 min	`android-e2e`	:robot:`apk` :calling:
:heavy_check_mark:	5bbc9d5a88bd906da62c96ae10c3ac43c64f6dda	#15	2024-05-21 16:58:39	~8 min	`android`	:robot:`apk` :calling:
:heavy_check_mark:	5bbc9d5a88bd906da62c96ae10c3ac43c64f6dda	#15	2024-05-21 16:59:13	~8 min	`ios`	:iphone:`ipa` :calling:

:heavy_check_mark:	1a6d3a3b3e0e78bb4d28252563a37b550c77d75e	#16	2024-05-21 18:05:39	~4 min	`tests`	:page_facing_up:`log`
:heavy_check_mark:	1a6d3a3b3e0e78bb4d28252563a37b550c77d75e	#16	2024-05-21 18:09:34	~8 min	`android-e2e`	:robot:`apk` :calling:
:heavy_check_mark:	1a6d3a3b3e0e78bb4d28252563a37b550c77d75e	#16	2024-05-21 18:09:41	~8 min	`android`	:robot:`apk` :calling:
:heavy_check_mark:	1a6d3a3b3e0e78bb4d28252563a37b550c77d75e	#16	2024-05-21 18:10:08	~8 min	`ios`	:iphone:`ipa` :calling:

:heavy_check_mark:	5818b9f0672011deaa964b8111f6a38919e72e2d	#17	2024-05-22 17:56:09	~4 min	`tests`	:page_facing_up:`log`
:heavy_check_mark:	5818b9f0672011deaa964b8111f6a38919e72e2d	#17	2024-05-22 18:00:17	~8 min	`android-e2e`	:robot:`apk` :calling:
:heavy_check_mark:	5818b9f0672011deaa964b8111f6a38919e72e2d	#17	2024-05-22 18:00:23	~8 min	`android`	:robot:`apk` :calling:
:heavy_check_mark:	5818b9f0672011deaa964b8111f6a38919e72e2d	#17	2024-05-22 18:00:29	~8 min	`ios`	:iphone:`ipa` :calling:

:heavy_check_mark:	7a35b182e840bdd54622ec2d2336fed2578eb921	#18	2024-05-23 16:18:57	~4 min	`tests`	:page_facing_up:`log`
:heavy_check_mark:	7a35b182e840bdd54622ec2d2336fed2578eb921	#18	2024-05-23 16:22:43	~8 min	`android-e2e`	:robot:`apk` :calling:
:heavy_check_mark:	7a35b182e840bdd54622ec2d2336fed2578eb921	#18	2024-05-23 16:22:46	~8 min	`android`	:robot:`apk` :calling:
:heavy_check_mark:	7a35b182e840bdd54622ec2d2336fed2578eb921	#18	2024-05-23 16:23:07	~8 min	`ios`	:iphone:`ipa` :calling:

:grey_question:	Commit	:hash:	Finished (UTC)	Duration	Platform	Result
:heavy_check_mark:	cc1a9ad8ba1828029207648208ad611636551059	#19	2024-05-24 15:23:23	~8 min	`android`	:robot:`apk` :calling:
:heavy_check_mark:	cc1a9ad8ba1828029207648208ad611636551059	#19	2024-05-24 15:23:23	~8 min	`android-e2e`	:robot:`apk` :calling:
:heavy_check_mark:	cc1a9ad8ba1828029207648208ad611636551059	#19	2024-05-24 15:23:53	~8 min	`ios`	:iphone:`ipa` :calling:
:heavy_check_mark:	cc1a9ad8ba1828029207648208ad611636551059	#20	2024-05-24 15:42:32	~3 min	`tests`	:page_facing_up:`log`

:heavy_check_mark:	1e50005414367898971e4db4bffa94a48dd3b55b	#21	2024-05-24 15:49:25	~4 min	`tests`	:page_facing_up:`log`
:heavy_check_mark:	1e50005414367898971e4db4bffa94a48dd3b55b	#20	2024-05-24 15:53:29	~8 min	`android-e2e`	:robot:`apk` :calling:
:heavy_check_mark:	1e50005414367898971e4db4bffa94a48dd3b55b	#20	2024-05-24 15:53:36	~8 min	`android`	:robot:`apk` :calling:
:heavy_check_mark:	1e50005414367898971e4db4bffa94a48dd3b55b	#20	2024-05-24 15:54:00	~8 min	`ios`	:iphone:`ipa` :calling:

Apr 04 '24 22:04 status-im-auto

@OmarBasem - since this pr is around derivation paths and you have a lot of knowledge in that area, do you think you could give it a quick happy path test on the functionality added? Just install the pr build and report back if things worked/didn't work etc, nothing too strenuous of a test, like 5/ 10 mins checking 👍 If you don't have time today/next few days, perhaps someone else in @status-im/wallet-mobile-devs can take a look. (let's try limit ourselves to helping test 1 pr per week for the moment, so if you've already tested one / or are testing a pr then please let others volunteer)

Apr 05 '24 11:04 J-Son89

Thanks for your PR @ulisesmac!

I have tried testing out the PR. I tried to create a new keypair and a new account and it worked fine. But when I tried to create another account using one of the existing keypairs it did not work. After I enter the password it gets stuck -- nothing happens.

@OmarBasem thank you so much for spotting this issue!

Yeah, makes sense, since I'm not considering that case, I thought it wasn't implemented. I'll revisit the implementation and fix it.

Apr 05 '24 15:04 ulisesmac

Thanks for your PR @ulisesmac!

I have tried testing out the PR. I tried to create a new keypair and a new account and it worked fine. But when I tried to create another account using one of the existing keypairs it did not work. After I enter the password it gets stuck -- nothing happens. IMG_4506.MP4

Hey @OmarBasem @J-Son89 , a quick update on it, I'm currently working on it.

The new account generation based on a keypair is incomplete or buggy in develop. No matter if we pick any keypair, the code always generates an account for the root/main/primary? keypair stored, the one for our main account.

Besides the account creation, I think the keypair picker needs a lot of work, sometimes it's confusing what is being picked.

Screencast from 2024-04-08 17-42-34.webm

It took me a while to realize what the problems were and how to properly use it, as said before, I'm working on actually creating an account given a selected keypair.

TBH I believe this is kind of a separate issue, but I don't want to merge this implementation without being complete, also because it'll require an immediate follow-up, so I'm addressing it in this PR. :+1:

Apr 08 '24 23:04 ulisesmac

The new account generation based on a keypair is incomplete or buggy in develop.

@ulisesmac it is working fine for me on develop. It was QA'ed in this PR https://github.com/status-im/status-mobile/pull/19333. Creating account from new keypair or an existing keypair works as expected.

Apr 09 '24 09:04 OmarBasem

The new account generation based on a keypair is incomplete or buggy in develop.

@ulisesmac it is working fine for me on develop. It was QA'ed in this PR #19333. Creating account from new keypair or an existing keypair works as expected.

yeah it's a little unclear to me what issues exactly you are pointing to @ulisesmac ? 🤔 It's more design related?

Apr 09 '24 10:04 J-Son89

@OmarBasem @J-Son89

It's unclear for me what is the correct flow to derive an address for a keypair. I'll explain develop behaviour with a (replicable) example.

Expected(?) (Status Desktop behaves like this):

User adds account to Primary KP -> new acc in Primary KP with deriv. path m/44'/60'/0'/0/1
User adds account to Primary KP -> new acc in Primary KP with deriv. path m/44'/60'/0'/0/2
User creates a Secondary KP -> new acc in Secondary KP with deriv. path m/44'/60'/0'/0/0
User adds account to Secondary KP -> new acc in Secondary KP with deriv. path m/44'/60'/0'/0/1
User adds account to Secondary KP -> new acc in Secondary KP with deriv. path m/44'/60'/0'/0/2
User adds account to Primary KP -> new acc in Primary KP with deriv. path m/44'/60'/0'/0/3

Current (status mobile behavior):

User adds account to Primary KP -> new acc in Primary KP with deriv. path m/44'/60'/0'/0/1
User adds account to Primary KP -> new acc in Primary KP with deriv. path m/44'/60'/0'/0/2
User creates a Secondary KP -> new acc in Secondary KP with deriv. path m/44'/60'/0'/0/3 :warning:
User adds account to Secondary KP -> new acc in Secondary KP with deriv. path m/44'/60'/0'/0/4 :warning:
User adds account to Secondary KP -> new acc in Secondary KP with deriv. path m/44'/60'/0'/0/5 :warning:
User adds account to Primary KP -> new acc in Primary KP with deriv. path m/44'/60'/0'/0/6 :warning:

And all accounts marked with :warning: are generated in the same way we generate accounts for Primary KP, so the Secondary KP is just another set of Primary KP addresses. If we attempt to generate the missing m/44'/60'/0'/0/3 Primary KP address, we'll get an exception saying it has been already created.

Apr 09 '24 19:04 ulisesmac

@J-Son89 @OmarBasem Just to provide more context, I found this problem been already reported:

#19472

Apr 09 '24 20:04 ulisesmac

@ulisesmac I think you are referring to a different issue. Creating a new account from a new keypair or an existing keypair works as expected on develop. Whether the derivation path is accurate or not shouldn't affect the functionality.

Apr 15 '24 05:04 OmarBasem

85% of end-end tests have passed

Total executed tests: 52
Failed tests: 5
Expected to fail tests: 3
Passed tests: 44

IDs of failed tests: 727230,727229,702869,702730,727232

IDs of expected to fail tests: 703495,703503,702807

Failed tests (5)

Click to expand

Rerun failed tests

Class TestWalletOneDevice:

1. test_wallet_add_remove_watch_only_account, id: 727232

Device 1: `Text` is `0x8d2413447ff297d30bdc475f6d5cb00254685aae`
Device 1: Click system back button

critical/test_wallet.py:213: in test_wallet_add_remove_watch_only_account
    self.home_view.driver.fail(
base_test_case.py:178: in fail
    pytest.fail('Device %s: %s' % (self.number, text))
 Device 1: Incorrect address '0x8d2413447ff297d30bdc475f6d5cb00254685aae' is shown when swiping between accounts, expected one is '0x8d2413447ff297d30bdc475f6d5cb00254685aae'

Device sessions

Device 1:

Steps, video, logs

Class TestWalletMultipleDevice:

1. test_wallet_send_asset_from_drawer, id: 727230

critical/test_wallet.py:119: in test_wallet_send_asset_from_drawer
    self.wallet_view.navigate_back_to_wallet_view()
 'TestWalletMultipleDevice' object has no attribute 'wallet_view'

2. test_wallet_send_eth, id: 727229

Device 1: Swiping right on element SlideButton
Device 1: Find SlideButton by xpath: //*[@resource-id='slide-button-track']

critical/test_wallet.py:111: in test_wallet_send_eth
    self.wallet_1.send_asset(address=self.receiver['address'], asset_name='Ether', amount=amount_to_send)
../views/wallet_view.py:100: in send_asset
    self.confirm_transaction()
../views/wallet_view.py:87: in confirm_transaction
    self.slide_and_confirm_with_password()
../views/wallet_view.py:81: in slide_and_confirm_with_password
    self.slide_button_track.slide()
../views/base_view.py:257: in slide
    self.swipe_right_on_element(width_percentage=1.3, start_x=100)
../views/base_element.py:308: in swipe_right_on_element
    location, size = self.get_element_coordinates()
../views/base_element.py:294: in get_element_coordinates
    element = self.find_element()
../views/base_element.py:79: in find_element
    raise NoSuchElementException(
 Device 1: SlideButton by xpath: `//*[@resource-id='slide-button-track']` is not found on the screen; For documentation on this error, please visit: https://www.selenium.dev/documentation/webdriver/troubleshooting/errors#no-such-element-exception

Class TestCommunityOneDeviceMerged:

1. test_community_undo_delete_message, id: 702869

Device 1: Tap on found: Button
Device 1: Find Button by xpath: //*[@text="Undo"]

critical/chats/test_public_chat_browsing.py:112: in test_community_undo_delete_message
    self.channel.element_by_text("Undo").click()
../views/base_element.py:92: in click
    element.click()
../../../../status-app-prs@tmp/venv/lib/python3.10/site-packages/selenium/webdriver/remote/webelement.py:94: in click
    self._execute(Command.CLICK_ELEMENT)
../../../../status-app-prs@tmp/venv/lib/python3.10/site-packages/selenium/webdriver/remote/webelement.py:395: in _execute
    return self._parent.execute(command, params)
../../../../status-app-prs@tmp/venv/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py:345: in execute
    self.error_handler.check_response(response)
../../../../status-app-prs@tmp/venv/lib/python3.10/site-packages/appium/webdriver/errorhandler.py:122: in check_response
    raise exception_class(msg=message, stacktrace=format_stacktrace(stacktrace))
 The element 'By.xpath: //*[@text="Undo"]' does not exist in DOM anymore; For documentation on this error, please visit: https://www.selenium.dev/documentation/webdriver/troubleshooting/errors#stale-element-reference-exception
E   Stacktrace:
E   io.appium.uiautomator2.common.exceptions.StaleElementReferenceException: The element 'By.xpath: //*[@text="Undo"]' does not exist in DOM anymore
E   	at io.appium.uiautomator2.model.ElementsCache.restore(ElementsCache.java:117)
E   	at io.appium.uiautomator2.model.ElementsCache.get(ElementsCache.java:153)
E   	at io.appium.uiautomator2.handler.Click.safeHandle(Click.java:36)
E   	at io.appium.uiautomator2.handler.request.SafeRequestHandler.handle(SafeRequestHandler.java:59)
E   	at io.appium.uiautomator2.server.AppiumServlet.handleRequest(AppiumServlet.java:277)
E   	at io.appium.uiautomator2.server.AppiumServlet.handleHttpRequest(AppiumServlet.java:271)
E   	at io.appium.uiautomator2.http.ServerHandler.channelRead(ServerHandler.java:68)
E   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
E   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
E   	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345)
E   	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
E   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
E   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
E   	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345)
E   	at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:435)
E   	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:293)
E   	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:267)
E   	at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:250)
E   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
E   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
E   	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345)
E   	at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:266)
E   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
E   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
E   	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345)
E   	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1294)
E   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
E   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
E   	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:911)
E   	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131)
E   	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:611)
E   	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:552)
E   	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:466)
E   	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:438)
E   	at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:140)
E   	at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
E   	at java.lang.Thread.run(Thread.java:1012)

During handling of the above exception, another exception occurred:
critical/chats/test_public_chat_browsing.py:114: in test_community_undo_delete_message
    pytest.fail("Can't press Undo button, not enough time")
E   Failed: Can't press Undo button, not enough time

Device sessions

Device 1:

Steps, video, logs

Class TestOneToOneChatMultipleSharedDevicesNewUi:

1. test_1_1_chat_message_reaction, id: 702730

Device 2: Find EmojisNumber by xpath: //*[starts-with(@text,'Message sender')]/ancestor::android.view.ViewGroup[@content-desc='chat-item']/../..//*[@content-desc='emoji-reaction-1']/android.widget.TextView[2]
Device 2: Find EmojisNumber by xpath: //*[starts-with(@text,'Message sender')]/ancestor::android.view.ViewGroup[@content-desc='chat-item']/../..//*[@content-desc='emoji-reaction-1']/android.widget.TextView[2]

critical/chats/test_1_1_public_chats.py:86: in test_1_1_chat_message_reaction
    message_receiver.emojis_below_message(emoji="love").wait_for_element_text(2, 90)
../views/base_element.py:190: in wait_for_element_text
    self.driver.fail(message if message else "`%s` is not equal to expected `%s` in %s sec" % (
base_test_case.py:178: in fail
    pytest.fail('Device %s: %s' % (self.number, text))
 Device 2: `1` is not equal to expected `2` in 90 sec

Device sessions

Device 1:

Steps, video, logs

Device 2:

Steps, video, logs

Expected to fail tests (3)

Click to expand

Class TestGroupChatMultipleDeviceMergedNewUI:

1. test_group_chat_mute_chat, id: 703495

# STEP: Change device time so chat will be unmuted by timer
Device 2: Long press on ChatElement

critical/chats/test_group_chat.py:466: in test_group_chat_mute_chat
    self.errors.verify_no_errors()
base_test_case.py:190: in verify_no_errors
    pytest.fail('\n '.join([self.errors.pop(0) for _ in range(len(self.errors))]))
 Chat is still muted after timeout

[[Chat is not unmuted after expected time: https://github.com/status-im/status-mobile/issues/19627]]

Device sessions

Device 1:

Steps, video, logs

Device 2:

Steps, video, logs

Device 3:

Steps, video, logs

2. test_group_chat_join_send_text_messages_push, id: 702807

Device 2: Find Text by xpath: //*[starts-with(@text,'Hey, admin!')]/ancestor::android.view.ViewGroup[@content-desc='chat-item']//*[@content-desc='message-status']/android.widget.TextView
Device 2: Text is Sent

critical/chats/test_group_chat.py:97: in test_group_chat_join_send_text_messages_push
    self.chats[1].chat_element_by_text(message_to_admin).wait_for_status_to_be('Delivered', timeout=120)
../views/chat_view.py:225: in wait_for_status_to_be
    raise TimeoutException("Message status was not changed to %s, it's %s" % (expected_status, current_status))
 Message status was not changed to Delivered, it's Sent

[[Issue with a message status - Sent instead of Delivered, https://github.com/status-im/status-mobile/issues/20126]]

Device sessions

Device 1:

Steps, video, logs

Device 2:

Steps, video, logs

Device 3:

Steps, video, logs

Class TestCommunityOneDeviceMerged:

1. test_community_discovery, id: 703503

Test is not run, e2e blocker

[[reason: [NOTRUN] Curated communities not loading, https://github.com/status-im/status-mobile/issues/17852]]

Passed tests (44)

Click to expand

Class TestActivityMultipleDevicePRTwo:

1. test_activity_center_mentions, id: 702957

Device sessions

Device 1:

Steps, video, logs

Device 2:

Steps, video, logs

2. test_activity_center_admin_notification_accept_swipe, id: 702958

Device sessions

Device 1:

Steps, video, logs

Device 2:

Steps, video, logs

Class TestWalletOneDevice:

1. test_wallet_add_remove_regular_account, id: 727231

Device sessions

Device 1:

Steps, video, logs

Class TestOneToOneChatMultipleSharedDevicesNewUiTwo:

1. test_1_1_chat_delete_via_long_press_relogin, id: 702784

Device sessions

Device 1:

Steps, video, logs

Device 2:

Steps, video, logs

2. test_1_1_chat_is_shown_message_sent_delivered_from_offline, id: 702783

Device sessions

Device 1:

Steps, video, logs

Device 2:

Steps, video, logs

3. test_1_1_chat_mute_chat, id: 703496

Device sessions

Device 1:

Steps, video, logs

Device 2:

Steps, video, logs

Class TestGroupChatMultipleDeviceMergedNewUI:

1. test_group_chat_pin_messages, id: 702732

Device sessions

Device 1:

Steps, video, logs

Device 2:

Steps, video, logs

Device 3:

Steps, video, logs

2. test_group_chat_send_image_save_and_share, id: 703297

Device sessions

Device 1:

Steps, video, logs

Device 2:

Steps, video, logs

Device 3:

Steps, video, logs

3. test_group_chat_reactions, id: 703202

Device sessions

Device 1:

Steps, video, logs

Device 2:

Steps, video, logs

Device 3:

Steps, video, logs

4. test_group_chat_offline_pn, id: 702808

Device sessions

Device 1:

Steps, video, logs

Device 2:

Steps, video, logs

Device 3:

Steps, video, logs

Class TestActivityCenterContactRequestMultipleDevicePR:

1. test_add_contact_field_validation, id: 702777

Device sessions

Device 1:

Steps, video, logs

Device 2:

Steps, video, logs

2. test_activity_center_contact_request_accept_swipe_mark_all_as_read, id: 702851

Device sessions

Device 1:

Steps, video, logs

Device 2:

Steps, video, logs

3. test_activity_center_contact_request_decline, id: 702850

Device sessions

Device 1:

Steps, video, logs

Device 2:

Steps, video, logs

Class TestCommunityMultipleDeviceMergedTwo:

1. test_community_markdown_support, id: 702809

Device sessions

Device 1:

Steps, video, logs

Device 2:

Steps, video, logs

2. test_community_hashtag_links_to_community_channels, id: 702948

Device sessions

Device 1:

Steps, video, logs

Device 2:

Steps, video, logs

3. test_community_mentions_push_notification, id: 702786

Device sessions

Device 1:

Steps, video, logs

Device 2:

Steps, video, logs

4. test_community_leave, id: 702845

Device sessions

Device 1:

Steps, video, logs

Device 2:

Steps, video, logs

5. test_community_join_when_node_owner_offline, id: 703629

Device sessions

Device 1:

Steps, video, logs

Device 2:

Steps, video, logs

Class TestDeepLinksOneDevice:

1. test_links_open_universal_links_from_chat, id: 704613

Device sessions

Device 1:

Steps, video, logs

2. test_links_deep_links, id: 702775

Device sessions

Device 1:

Steps, video, logs

Class TestCommunityOneDeviceMerged:

1. test_restore_multiaccount_with_waku_backup_remove_switch, id: 703133

Device sessions

Device 1:

Steps, video, logs

2. test_community_copy_and_paste_message_in_chat_input, id: 702742

Device sessions

Device 1:

Steps, video, logs

3. test_community_navigate_to_channel_when_relaunch, id: 702846

Device sessions

Device 1:

Steps, video, logs

4. test_community_mute_community_and_channel, id: 703382

Device sessions

Device 1:

Steps, video, logs

Class TestOneToOneChatMultipleSharedDevicesNewUi:

1. test_1_1_chat_emoji_send_reply_and_open_link, id: 702782

Device sessions

Device 1:

Steps, video, logs

Device 2:

Steps, video, logs

2. test_1_1_chat_text_message_delete_push_disappear, id: 702733

Device sessions

Device 1:

Steps, video, logs

Device 2:

Steps, video, logs

3. test_1_1_chat_push_emoji, id: 702813

Device sessions

Device 1:

Steps, video, logs

Device 2:

Steps, video, logs

4. test_1_1_chat_non_latin_messages_stack_update_profile_photo, id: 702745

Device sessions

Device 1:

Steps, video, logs

Device 2:

Steps, video, logs

5. test_1_1_chat_edit_message, id: 702855

Device sessions

Device 1:

Steps, video, logs

Device 2:

Steps, video, logs

6. test_1_1_chat_send_image_save_and_share, id: 703391

Device sessions

Device 1:

Steps, video, logs

Device 2:

Steps, video, logs

7. test_1_1_chat_pin_messages, id: 702731

Device sessions

Device 1:

Steps, video, logs

Device 2:

Steps, video, logs

Class TestCommunityMultipleDeviceMerged:

1. test_community_several_images_send_reply, id: 703194

Device sessions

Device 1:

Steps, video, logs

Device 2:

Steps, video, logs

2. test_community_one_image_send_reply, id: 702859

Device sessions

Device 1:

Steps, video, logs

Device 2:

Steps, video, logs

3. test_community_emoji_send_copy_paste_reply, id: 702840

Device sessions

Device 1:

Steps, video, logs

Device 2:

Steps, video, logs

4. test_community_mark_all_messages_as_read, id: 703086

Device sessions

Device 1:

Steps, video, logs

Device 2:

Steps, video, logs

5. test_community_contact_block_unblock_offline, id: 702894

Device sessions

Device 1:

Steps, video, logs

Device 2:

Steps, video, logs

6. test_community_edit_delete_message_when_offline, id: 704615

Device sessions

Device 1:

Steps, video, logs

Device 2:

Steps, video, logs

7. test_community_message_delete, id: 702839

Device sessions

Device 1:

Steps, video, logs

Device 2:

Steps, video, logs

8. test_community_message_send_check_timestamps_sender_username, id: 702838

Device sessions

Device 1:

Steps, video, logs

Device 2:

Steps, video, logs

9. test_community_links_with_previews_github_youtube_twitter_gif_send_enable, id: 702844

Device sessions

Device 1:

Steps, video, logs

Device 2:

Steps, video, logs

10. test_community_message_edit, id: 702843

Device sessions

Device 1:

Steps, video, logs

Device 2:

Steps, video, logs

11. test_community_unread_messages_badge, id: 702841

Device sessions

Device 1:

Steps, video, logs

Device 2:

Steps, video, logs

Class TestActivityMultipleDevicePR:

1. test_navigation_jump_to, id: 702936

Device sessions

Device 1:

Steps, video, logs

Device 2:

Steps, video, logs

2. test_activity_center_reply_read_unread_delete_filter_swipe, id: 702947

Device sessions

Device 1:

Steps, video, logs

Device 2:

Steps, video, logs

May 21 '24 19:05 status-im-auto

Hi @ulisesmac, thanks a lot for the fixes! I started testing and now need your clarification/confirmation on the fix for https://github.com/status-im/status-mobile/issues/19232 and how we should generate a derivation path for a newly created account after removing a previous one.

Case:

Create a new profile
Go to wallet > Add some new derived accounts for the existing (default) keypair, e.g Acc1 (derivation path m/44'/60'/0'/0/1) and Acc2 (m/44'/60'/0'/0/2)
Remove Acc1 (m/44'/60'/0'/0/1)
Add Acc3

The expected result in the issue is saying:

"The Derivation path for the newly created account should be the same as it was when the account was removed" i.e. in our case, it should be m/44'/60'/0'/0/1 for Acc3

Could you please confirm, that is NOT correct?

IIRC we asked you about this on the offsite, and your answer was that if the account is removed, we should continue to generate new accounts with derivation paths in the same order, that is for Acc3 it should be m/44'/60'/0'/0/3 (and that's how it works both in nightly and PR)

I'm asking for a clarification again because the description of this PR indicates that https://github.com/status-im/status-mobile/issues/19232 is fixed and you mentioned there that

The derivation path for derived accounts is incremented correctly, even if a derived account is removed.

However, I don't see any changes in this PR compared to nightly. And on the other hand, it looks expected to me.

PR:

https://github.com/status-im/status-mobile/assets/67952253/f02053f6-c974-4dc1-87ce-8efa7b23ed7c

Does it mean that only the part of the issue where the UI froze when creating a new account was fixed? Please help me solve this puzzle :)

I'll update the issue description if needed just to make it clear. Thanks!

May 22 '24 14:05 qoqobolo

Hi @qoqobolo Thanks for testing!

Sorry for not being clear enough in this PR, I'll explain it with more detail.

I started testing and now need your clarification/confirmation on the fix for #19232 and how we should generate a derivation path for a newly created account after removing a previous one.

Case:
1. Create a new profile

2. Go to wallet > Add some new derived accounts for the existing (default) keypair, e.g Acc1 (derivation path m/44'/60'/0'/0/1) and Acc2 (m/44'/60'/0'/0/2)

3. Remove Acc1 (m/44'/60'/0'/0/1)

4. Add Acc3

Yeah, I think these steps aren't not enough to replicate the issue

The expected result in the issue is saying:

"The Derivation path for the newly created account should be the same as it was when the account was removed" i.e. in our case, it should be m/44'/60'/0'/0/1 for Acc3

Could you please confirm, that is NOT correct?

Exactly, this is NOT correct. The expected result should be m/44'/60'/0'/0/3 for Acc 3

IIRC we asked you about this on the offsite, and your answer was that if the account is removed, we should continue to generate new accounts with derivation paths in the same order, that is for Acc3 it should be m/44'/60'/0'/0/3 (and that's how it works both in nightly and PR)

Yeah, m/44'/60'/0'/0/3 should be the next expected derivation path, I confirmed it with Alisher some weeks ago.

and that's how it works both in nightly and PR

Maybe that's because of the specific case you tested, but they don't behave the same, I recorded a video showing the error in the latest of develop:

Screencast from 2024-05-22 10-35-26.webm

And this is the same case in this PR:

Screencast from 2024-05-22 10-42-38.webm

Does it mean that only the part of the issue where the UI froze when creating a new account was fixed? Please help me solve this puzzle :)

Yes, I fixed the UI freezing, but the issue's description isn't correct, sorry, I should've updated it to properly describe the problem.

Now, regarding to this PR, the issue shown in the first video I shared becomes worse when we generate new keypairs, the derivation path just makes no sense when deriving new accounts for new keypairs. This is being fixed in this PR.

Additionally, in develop it looks as if we are generating new keypairs, but we are not, so this PR also fixes the keypair generation, this fix is very important, but I'm not sure what you can do to test that there ARE NOT new keypairs added in develop.

It can be seen as an internal fix in the app, the important thing is that now new keypairs and new derived accounts are working.

May 22 '24 17:05 ulisesmac

@ulisesmac thank you very much for the detailed explanation! Everything is clear and I see that the issue is deeper than it seemed before

Additionally, in develop it looks as if we are generating new keypairs, but we are not, so this PR also fixes the keypair generation, this fix is very important, but I'm not sure what you can do to test that there ARE NOT new keypairs added in develop.

Yes, unfortunately, I don't know either but if you checked it yourself, it should be enough.

Regarding testing the PR, I've checked the following cases:

removing Acc 2 and adding Acc 4 (that we mentioned above)
creating a new key pair and adding new derived accounts for it (without removing them due to https://github.com/status-im/status-mobile/issues/19375)
cases 1 and 2 on a profile recovered from a seed phrase
cases 1 and 2 on synced devices (there is an interesting issue: if you create a new keypair on a primary device, you can't add a new derived account for it on a secondary device. But it also exists in develop, I'll investigate and log it separately)

What I couldn't test is the same cases for the Import keypair using recovery phrase scenario. Please check this question on Discord when you're online, will wait for your answer (in case you have some context) and log it as well if needed.

And let me know if you have smth else in mind that I can test. Thank you!

May 23 '24 12:05 qoqobolo

4. cases 1 and 2 on synced devices (there is an interesting issue: if you create a new keypair on a primary device, you can't add a new derived account for it on a secondary device. But it also exists in develop, I'll investigate and log it separately)

This one is very interesting, I guess that's because you need to provide the seed phrase for every new keypair you added. and probably you are not able to send transactions too.

I think those flows are being covered by the Wallet Settings team, but just guessing that's the issue.

CC: @smohamedjavid any idea of what might be the issue here?

May 23 '24 16:05 ulisesmac

What I couldn't test is the same cases for the Import keypair using recovery phrase scenario. Please check this question on Discord when you're online, will wait for your answer (in case you have some context) and log it as well if needed.

@qoqobolo

Yeah, we already clarified that and we'll open an issue for it, basically it wasn't properly updated in a past PR and it also needs some fixes that are being done in this PR.

The test cases are good enough, thanks for taking the time! :+1:

May 23 '24 16:05 ulisesmac

You are right @ulisesmac.

cases 1 and 2 on synced devices (there is an interesting issue: if you create a new keypair on a primary device, you can't add a new derived account for it on a secondary device. But it also exists in develop, I'll investigate and log it separately)

@qoqobolo - If the primary device creates any new keypair, the secondary device will get only the key pair details without the keystore file. The secondary device should import the missing keypair from the device it was created (primary device in our case) or import using the Recovery phrase or Private key to perform any transactions on those accounts.

P.S. A sneak peek of the feature. We can check the missing keypairs in settings profile > wallet > key pairs and accounts. We need to enable the feature flag keypairs-and-accounts under settings.

May 23 '24 16:05 smohamedjavid

@ulisesmac @smohamedjavid thanks for your clarifications guys!

@ulisesmac no issues from my side, PR can be merged, thanks again for the amazing work!

May 24 '24 08:05 qoqobolo

@ulisesmac @qoqobolo thank you for providing clear explanations, it really matters in particular case!

Amazing work!!!

May 24 '24 09:05 churik