CodeEdit icon indicating copy to clipboard operation
CodeEdit copied to clipboard

Editor View Tests

Open thecoolwinter opened this issue 1 year ago • 2 comments

Description

Creates tests for the editor view, ~this is a prerequisite PR for the fix for #1545.~ This also adds back a few tests that were previously removed, ~and adds content to the CodeEditUITests target.~ I've also added a few helper functions for future UI tests that may require snapshot testing or a test workspace to use.

This PR now removes the UITests target. This should be added back in the future when a solution can be found for our test runner. That future PR can be considered a completion of this PR, but this one still solves a major issue with our tests right now.

It's also important to note the separation between tests. There's tests for the editor view in both the CodeEditTests target and the CodeEditUITests target. The CodeEditTests target performs tests that can be done without user interaction. For example, creating an editor layout with a certain selection or split layout is done in this target. However for testing the focus button and that focusing a tab correctly displays in a workspace environment, the CodeEditUITests target is used as it can perform interaction.

~I've also added small helpers in the CodeEditUITests target to do two things:~ ~- Disable sandboxing in the UI Testing app (to allow snapshots to be saved and accessed)~ ~- Create a consistent screenshot of an element query (often the snapshot method returns inconsistently sized images that appear exactly the same but are different).~

Also in this PR:

  • Misc additions to various components to add accessibility identifiers to hopefully improve UI Testing readability in the future. These (for the most part) are not visible to the end user. With the exception of a few elements that needed an accessibility label.
  • Created an extension on CommandLine for checking if the app is being run in debug mode. These checks are only enabled in a debug build. These options should be used in interactive UI tests when they're added back.
  • While making that extension, moved some code into an extension from AppDelegate to clean it up.
  • Added back WelcomeModule tests

A note about the previous draft status

Instead of waiting for a (as of writing) 3 year old PR to be merged into the snapshot testing library, I've written a small wrapper that generates a consistent image for an NSView or SwiftUI View regardless of what resolution a machine is running on. This results in snapshots that are 1/2 the quality as they were before, but will be deterministic moving forward.

Related Issues

  • #1545

Checklist

  • [x] I read and understood the contributing guide as well as the code of conduct
  • [x] The issues this PR addresses are related to each other
  • [x] My changes generate no new warnings
  • [x] My code builds and runs on my machine
  • [x] My changes are all related to the related issue above
  • [x] I documented my code

Screenshots

No UI Changes

thecoolwinter avatar Jan 15 '24 06:01 thecoolwinter

After further investigation, there's some issues with snapshotting NSViews with SwiftUI views. The reason tests are failing right now on this PR is the test runner runs at a different resolution than my machine. To fix this we need a different method for snapshotting NSViews. Either this could be merged into swift-snapshot-testing (or something like it) or we fork that repository and add the necessary requirements for our tests.

thecoolwinter avatar Jan 16 '24 15:01 thecoolwinter

Got a few tests passing 💪 I'll be able to pick this up in a few days

thecoolwinter avatar May 04 '24 19:05 thecoolwinter

Closing in favor of implementing automation tests now that #1804 is open. We can do snapshot tests for small components, and automation tests for workflows.

thecoolwinter avatar Jul 10 '24 21:07 thecoolwinter