jetpack icon indicating copy to clipboard operation
jetpack copied to clipboard
verified publisher icon Automattic

e2e tests: attempt to more explicitely start tunnel

Open jeherve opened this issue 7 months ago • 6 comments

Fixes #43260

Proposed changes:

Let's try to be more explicit in starting the tunnel for the test.

I'm not quite sure why tunnel.js on isn't enough ; let's try to fix that.

Other information:

  • [ ] Have you written new tests for your changes, if applicable?
  • [ ] Have you checked the E2E test CI results, and verified that your changes do not break them?
  • [ ] Have you tested your changes on WordPress.com, if applicable (if so, you'll see a generated comment below with a script to run)?

Jetpack product discussion

  • N/A

Does this pull request change what data or activity we track or use?

  • N/A

Testing instructions:

  • Is CI happy? https://github.com/Automattic/jetpack/actions/runs/14694210582/job/41233658493?pr=43261

jeherve avatar Apr 27 '25 16:04 jeherve

Thank you for your PR!

When contributing to Jetpack, we have a few suggestions that can help us test and review your patch:

  • :white_check_mark: Include a description of your PR changes.
  • :white_check_mark: Add a "[Status]" label (In Progress, Needs Review, ...).
  • :white_check_mark: Add a "[Type]" label (Bug, Enhancement, Janitorial, Task).
  • :white_check_mark: Add testing instructions.
  • :white_check_mark: Specify whether this PR includes any changes to data or privacy.
  • :white_check_mark: Add changelog entries to affected projects

This comment will be updated as you work on your PR and make changes. If you think that some of those checks are not needed for your PR, please explain why you think so. Thanks for cooperation :robot:

Follow this PR Review Process:

  1. Ensure all required checks appearing at the bottom of this PR are passing.
  2. Make sure to test your changes on all platforms that it applies to. You're responsible for the quality of the code you ship.
  3. You can use GitHub's Reviewers functionality to request a review.
  4. When it's reviewed and merged, you will be pinged in Slack to deploy the changes to WordPress.com simple once the build is done.

If you have questions about anything, reach out in #jetpack-developers for guidance!

github-actions[bot] avatar Apr 27 '25 16:04 github-actions[bot]

This CI job runs the performance tests from the Gutenberg repo?

It looks like you're now hitting upon the same error I encountered in #42934. The REST API utils used in some Gutenberg tests (usually for cleaning up things like posts created during the test) aren't working for some reason in the jetpack environment.

I had some thought's about what the issue might be here - https://github.com/Automattic/jetpack/pull/42934#issuecomment-2788209055.

But it still failed after I resolved that (see https://github.com/Automattic/jetpack/pull/42934/commits/8ebb283c9b1c593dd7fc481fbb9a76a9a64e6252). I haven't made an issue for this problem yet, I've been sidetracked on some other work.

talldan avatar Apr 28 '25 07:04 talldan

It looks like you're now hitting upon the same error I encountered in #42934.

Yup, it does indeed seem to be the same problem. @adimoldovan I know Woo also uses @wordpress/e2e-test-utils-playwright. Is that something you've run into in the past, by any chance?

jeherve avatar May 08 '25 18:05 jeherve

It looks like you're now hitting upon the same error I encountered in #42934.

Yup, it does indeed seem to be the same problem. @adimoldovan I know Woo also uses @wordpress/e2e-test-utils-playwright. Is that something you've run into in the past, by any chance?

No, sorry, this is the first time I'm seeing this error. I'll try setting up a local env, see if I can find anything.

adimoldovan avatar May 09 '25 08:05 adimoldovan

@jeherve I just dusted off #42934 again, rebased, and this time the tests passed, so perhaps my fix did work. 🤔

a775cc95ae6ed1db9cb4720cf7d0fc60d79f89d1 and e18dfa3b45c28a177c23031e8e875d5cb37a2a33 are the two commits to cherry-pick if you want to try it on this branch.

talldan avatar Jun 06 '25 08:06 talldan

Thanks for coming back to this PR with your findings!

It did seem to help, although we now seem to run into different issues further down the line: https://github.com/Automattic/jetpack/actions/runs/15489233312/job/43610754445?pr=43261

It seems the tests run successfully once, but then error out. @adimoldovan Do you know what could be happening here, maybe?

jeherve avatar Jun 06 '25 11:06 jeherve

Closing this for now since the test is now disabled, as per #44147

jeherve avatar Jul 01 '25 13:07 jeherve