wp-cli-tests icon indicating copy to clipboard operation
wp-cli-tests copied to clipboard

Published 20 hours ago • verified publisher icon wp-cli

Run tests against Windows

Open danielbachhuber opened this issue 2 years ago • 4 comments

To help prevent regressions like https://github.com/wp-cli/wp-cli/issues/5521, it would be nice to run our test suites against Windows.

Done is:

  • Our automated tests are running against Windows in all repos.

danielbachhuber avatar Oct 28 '22 17:10 danielbachhuber

As part of the nightly testing of the mentioned issue I've run the test suite on Windows which fail with the following error.

1) Extractor_Test::test_extract_tarball
Failed asserting that 1 is identical to 0.

wp-cli\tests\test-extractor.php:119

Tracing the error I find this.

tar: Option --force-local is not supported

Removing the ( Utils\is_windows() ? ' --force-local' : '' ) part of tar command within \Extractor_Test::test_extract_tarball makes all tests pass.

OK, but incomplete, skipped, or risky tests!
Tests: 270, Assertions: 722, Risky: 1.

I tested on Windows 10 against:

  1. PHP 7.4
  2. PHP 8.0
  3. PHP 8.1

mlipe-nxs avatar Oct 28 '22 20:10 mlipe-nxs

Removing the ( Utils\is_windows() ? ' --force-local' : '' ) part of tar command within \Extractor_Test::test_extract_tarball makes all tests pass.

Looks like that was added ~5 years ago: https://github.com/wp-cli/wp-cli/commit/271cd752f4bbd055bacbac3da01ca080d8b9363f#diff-ce006bf8351a3174bac26fef9078e893523d313ace23732b67ad97942e7b2ec4R115-R116

It was included in the same PR as the original issue: https://github.com/wp-cli/wp-cli/pull/4572

My guess is that @gitlost fixed some "earlier versions of PHP on Windows" issue that no longer exists. I don't know what the particular issue was or when it was resolved, though.

Pardon my extremely naive question but... which versions of Windows should we support?

danielbachhuber avatar Oct 31 '22 20:10 danielbachhuber

Great question... I believe the following are probably a reasonable place to start.

  1. Windows 10
  2. Windows 11
  3. Windows Server 2019
  4. Windows Server 2022

lipemat avatar Nov 01 '22 14:11 lipemat

No harm running tests against MacOS either, if it's easy to add to the matrix. I've had to fix differences in tar and sed. ...it's not unlikely that those same issues will reoccur under Windows. Unfortunately I don't have a regular Windows environment to test in (currently searching to see is Azure a tool I could use).

BrianHenryIE avatar Sep 12 '23 04:09 BrianHenryIE