go-mega icon indicating copy to clipboard operation
go-mega copied to clipboard

Add 2FA test and 2FA support for integration tests

Open FranciscoPombal opened this issue 1 year ago • 9 comments

Note: description updated according to updates below.


  • Credentials for non 2FA/MFA-enabled accounts as well as 2FA/MFA-enabled accounts are supplied from the environment.

  • ~For tests that can use both types of credentials, credentials for non 2FA/MFA-enabled accounts are preferred, if available.~

  • ~Tests that require one or the other type of credentials/accounts are skipped if the required type of credentials is not available.~

  • For the tests, a single MFA and non-MFA login is performed at the start of testing, and any subsequent logins for each test are done with session token logins.

  • A new test-only dependency is introduced for generating 2FA/MFA codes from their secrets: github.com/pquerna/[email protected].

  • golang.org/x/crypto has been bumped to the latest version.

  • Dropped usage of io/ioutil.


~This is not ready because I cannot get the tests to pass properly.~ ~With a 2FA/MFA-enabled account, I can get all tests to pass except one if I:~

  • ~Run them one at a time.~
  • ~Wait a bit between running each test (a few tens of seconds).~
  • ~Clear the login sessions created by the tests in the MEGA web app's "Session history" interface between each test.~

~When these workarounds are not all performed, the tests will fail with the following error: The upload target URL you are trying to access has expired. Please request a fresh one. Additionally, TestEventNotify is not passing no matter what, with the same error.~

I have not tested with a non 2FA/MFA-enabled account.


Currently my knowledge of MEGA's API and SDK internals is limited, so any help fixing these issues is appreciated.

~I would expect we need to at least clean up sessions between tests (perform a logout).~ A logout is performed on testing teardown.


Test setup to replicate the above results (for a 2FA/MFA-enabled account):

  1. Export these variables to the environment:

    [email protected]
    MEGA_PASSWD_MFA=foobar123
    MEGA_SECRET_MFA=SECRETTOKEN2FAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA====
    
  2. Run tests.

    Note that you may need to increase go test's -timeout, especially if the MEGA account used for testing has many files, because for such accounts it may take a long time to login and/or perform other operations under test.

FranciscoPombal avatar Aug 09 '24 17:08 FranciscoPombal

I would expect we need to at least clean up sessions between tests (perform a logout).

We really should be doing this anyway as the sessions just pile up. Rclone now has a Shutdown method which could be used for this.

Currently my knowledge of MEGA's API and SDK internals is limited, so any help fixing these issues is appreciated. I would expect we need to at least clean up sessions between tests (perform a logout).

My knowledge is limited too! I took on a maintainer role for this repo so I could keep it working with rclone, but I don't have time to dig into the SDK to find out how stuff works. I think you know more about it than me now!

These tests look safe to merge as they won't run unless we have the correct env var.

ncw avatar Aug 12 '24 14:08 ncw

OK I've implemented the logout command. Now tests won't leave sessions behind, if they finish successfully, and if the logout request at the end of each test is performed successfully. Tests are still leaving behind test files and directories, though. Session management in tests can be made more robust still, but it's a start.

Additionally, I now understand why the tests are not passing unless run one at a time, waiting some time between each test.

At least with MFA-enabled accounts, the current way go-mega does logins will cause all logins after a first one to fail if the following logins are done with the same 2FA code as the first one, even if the initial session is terminated properly (logged out). This means one must wait at most 30 seconds before running the next test (the time for a 2FA code to expire).

I need to look at the SDK and official apps/web app some more to figure out how they handle the login flow to account for this.

FranciscoPombal avatar Sep 16 '24 22:09 FranciscoPombal

Great progress @FranciscoPombal :-)

Let me know when you think you'd like to merge what you've done.

ncw avatar Sep 17 '24 15:09 ncw

Thanks! I have submitted an issue to the MEGA SDK repo with my findings of what I believe to be an issue in the SDK or maybe server-side. If they address this it would make our lives a lot easier.

Regardless, I have an idea for how to work around the remaining blockers for this PR even if nothing comes out of that report; we'll see.

FranciscoPombal avatar Sep 17 '24 16:09 FranciscoPombal

OK, I've implemented session logins to work around the login issues.

Tests are all passing now, except for one that is explicitly skipped, TestEventNotify, which I've noted in the relevant commit message. I think it should be dealt with later.

So this PR is ready!

FranciscoPombal avatar Sep 20 '24 10:09 FranciscoPombal

I've just noticed that as it stands, this PR supersedes https://github.com/t3rm1n4l/go-mega/pull/36 (thus closing https://github.com/t3rm1n4l/go-mega/issues/35) and https://github.com/t3rm1n4l/go-mega/pull/47.

https://github.com/t3rm1n4l/go-mega/pull/30 also appears to have been obsoleted already. https://github.com/t3rm1n4l/go-mega/issues/25 should be closed as well, since 2FA login is already supported by this library; with regards to rclone specifically, it duplicates https://github.com/rclone/rclone/issues/3165.

FranciscoPombal avatar Sep 20 '24 23:09 FranciscoPombal

@ncw ping

FranciscoPombal avatar Sep 25 '24 10:09 FranciscoPombal

Wow, great results!

vinibali avatar Nov 16 '24 04:11 vinibali

I've been following the request for this feature in the rclone issue for literally years! Thanks so much for your work on this @FranciscoPombal!!

bmacphail avatar Nov 20 '24 06:11 bmacphail

@ncw @FranciscoPombal - what happened to this PR? Looks like progress just stalled out. The conflicts look easy enough to merge.

brokosz avatar Jul 31 '25 05:07 brokosz

Probably this issue is a lower priority because MEGA now provides S3-compatible object storage, which can be used with 2FA enabled on the account and oblivates the need to use this library for backup tools like restic or rclone.

On July 30, 2025 10:16:11 PM PDT, Brad Rokosz @.***> wrote:

brokosz left a comment (t3rm1n4l/go-mega#51)

@ncw @FranciscoPombal - what happened to this PR? Looks like progress just stalled out. The conflicts look easy enough to merge.

-- Reply to this email directly or view it on GitHub: https://github.com/t3rm1n4l/go-mega/pull/51#issuecomment-3138587820 You are receiving this because you commented.

Message ID: @.***>

bmacphail avatar Jul 31 '25 13:07 bmacphail

Probably this issue is a lower priority because MEGA now provides S3-compatible object storage, which can be used with 2FA enabled on the account and oblivates the need to use this library for backup tools like restic or rclone.

Indeed, and it looks like this project is abandoned or will soon be. Closing this.

FranciscoPombal avatar Aug 09 '25 17:08 FranciscoPombal