wp-cli-tests icon indicating copy to clipboard operation
wp-cli-tests copied to clipboard

Add unified diff output to failed Behat comparisons

Open Copilot opened this issue 2 weeks ago • 4 comments
trafficstars

Summary

Successfully improved Behat output to include diffs on failed comparisons, making it much easier to debug test failures.

Completed Tasks

  • [x] Add sebastian/diff v3 as a dependency (compatible with PHP 7.2+)
  • [x] Create a helper method to generate diffs with proper type hints
  • [x] Update check_string method to show diffs when comparisons fail
  • [x] Update compare_tables method to show diffs
  • [x] Update JSON comparison methods to show diffs with proper error checking
  • [x] Update CSV comparison method to show diffs
  • [x] Update YAML comparison method to show diffs
  • [x] Add parameter and return type hints
  • [x] Add error checking for JSON encoding/decoding
  • [x] Fix pre-existing bug in CSV comparison variable usage
  • [x] Fix typo in YAML docstring
  • [x] Verify PHP syntax is correct
  • [x] Run code review and address all feedback
  • [x] Run security checks
  • [x] Update to use sebastian/diff v3 API for PHP 7.2+ compatibility

Changes Summary

3 files changed, 78 insertions(+), 13 deletions(-)

  1. composer.json: Added sebastian/diff v3 as an explicit dependency
  2. src/Context/Support.php: Added generate_diff() helper method using v3 API
  3. src/Context/ThenStepDefinitions.php: Enhanced all comparison methods to show diffs

Example Output

When a comparison fails, users will now see:

--- Expected
+++ Actual
@@ -1,3 +1,3 @@
 Hello World
-This is a test
+This is different
 Third line

Quality Assurance

✅ No PHP syntax errors ✅ No security vulnerabilities ✅ All code review feedback addressed ✅ Proper error handling for JSON operations ✅ Full type hints added ✅ Compatible with PHP 7.2+

Ready for merge!

Original prompt

This section details on the original issue you should resolve

<issue_title>Improve Behat output to include diffs on failed comparisons</issue_title> <issue_description>When Behat compares two elements and finds them to not match, it would be helpful for debugging purposes to have a diff available that shows the exact difference between the two elements, like in this example output:

image 2018-08-16 at 9 21 38 am

The SebastianBergmann\Diff\Differ class is available and can be used for that.</issue_description>

<agent_instructions>https://github.com/Behat/Behat/pull/1532 might be useful as inspiration.</agent_instructions>

Comments on the Issue (you are @copilot in this section)

@swissspidy Looks like an upcoming Behat release will have better diffs out of the box, see https://github.com/Behat/Behat/pull/1532

Not in v3.16 yet, so have to wait for the next one.

Requires PHP 7.2+ though, see https://github.com/wp-cli/wp-cli-tests/pull/228

Alternatively we can copy that logic of course.</comment_new>

  • Fixes wp-cli/wp-cli-tests#17

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot avatar Nov 01 '25 19:11 Copilot