Expose validation status digest (in admin bar) when DevTools disabled

[amedina]

In 2.0 when dev tools are hidden, the user does not have a way to get any information about the validation status of their pages/site. This is problematic in cases when, for example, some invalid markup is introduced by a plugin, the AMP plugins removes it and there is some misbehavior of the site and the user has no ides what is that coming from.

The solution is to provide a middle point between having full validation tools/information exposure to the user, and having none. When in Reader mode for non-tech-savvy users, the plugin should provide a "digest" of validation status to the user as an indication that there are issues.

---

Do not alter or remove anything below. The following sections will be managed by moderators only.

Acceptance criteria

Implementation brief

QA testing instructions

Demo

Changelog entry

[westonruter]

Is this digest the same as shown in the admin bar dropdown as added on #4986?

[amedina]

That is more detailed than I had in mind. Right now is zero information. A mid point between that and dev tools on would be something like "Your site has validation issues". And perhaps a notice (in admin screen maybe) to either turn dev tools on or contact the system admin, or disable offending plugins.

[westonruter]

If the user is an admin, there is no need to turn on dev tools as they can still access the validation screens even with dev tools turned off.

For users who are not admins, then that is where we need to mention something but tell them to ask an admin for help, since they would not be able to access dev tools.

[westonruter]

In this way it's related to https://github.com/ampproject/amp-wp/issues/3821#issuecomment-669536348.

[schlessera]

We should change the way we refer to the change we make to the DevTools. What we need is feedback on validation errors for users, even when DevTools is "hidden". However, adding feedback then would basically create a second, simplified DevTools mode that replaces the one we just hid.

Instead of "hiding" or "disabling" DevTools for non-developers, I think we should rephrase this in the sense that we "add a simplified mode" to DevTools.

So what we have now would be the "advanced mode" of DevTools, and for non-developers, we'd add a stripped-down or simplified version, that only focus on giving the most important bits to the user without overwhelming them.

[westonruter]

I'm not sure that we need to change how we refer to “disabling DevTools”. The simplified error view isn't really part of DevTools, I don't think. For example, in Chrome you can try going to a website with an bad HTTPS certificate and you get an error displayed on the page. You could also open DevTools to get more details about what the error is (I think), but users don't have to touch DevTools to be told about the error.

So I think what we need to do discontinue the exclusive coupling of validation errors to DevTools being enabled. When DevTools are disabled, we'd show the (minimized) errors but not reveal by default developer-focused technical bits that the user wouldn't understand. Instead of saying there is a validation error for an “Invalid script caused by plugin X” we should rather say there is a “Plugin incompatibility in X”. (See #4668.) Non-technical users could then have the needed information to rectify the problem, by switching to a different plugin or suppressing that plugin on AMP pages. There should then be a way to show the technical details (by going to the validated URL screen) or sending the error details to a technical administrator to investigate.

[westonruter]

Re-designating this in the UX workstream. As mentioned in https://github.com/ampproject/amp-wp/issues/2069#issuecomment-741522035 and as is being done in #5589 (related to #4668), I think the key for non-technical users is to surface the theme/plugins that are causing compatibility problems. That being the case, the digest I think should be a list of these components, in the same way that the block editor will be highlighting the theme/plugins responsible for the errors.

One issue here for showing this information in the admin bar is that the validation error source attribute is not available on normal frontend requests, so we'd have to say something initially like "There are AMP compatibility issues on this page." Clicking that menu item could then initiate a validation request to fetch the validation errors and their sources to then populate the in AMP admin bar menu.

cc @johnwatkins0 @jwold

[jwold]

Should we keep #5697 a separate issue, or merge the two? That feels like a sub-set of this one.

[westonruter]

Yes, keep separate. This issue is specifically for exposing the information in the admin bar, whereas that issue is for exposing it in the block editor sidebar.

[westonruter]

This depends on being able to initiate REST API request on the frontend. While this can be done now with the ?amp_validate query parameter, it would also be satisfied by #5552.

Note that the validation details should not be made available if the user does not have access to DevTools. If the user does have access, but they turned off Dev Tools in their user profile, then we can surface that information. Otherwise, non-privileged users would just be able to be told that there are validation issues that an administer needs to review.

For administrators with DevTools turned off, the admin bar item can show a warning sign as it does when DevTools are turned on. However, when DevTools are turned off, instead of "Validate URL" the menu item can just say "There are X validation issues."

In both cases, for users with DevTools enabled and disabled, I think we can then make this submenu item actually have a nested submenu. When that submenu is expanded, we can then show a "Loading" message and fetch the validation information for the current URL. We can then list out the theme/plugins which are causing the validation issues, or list out the validation errors in a similar way to they are presented in the AMP Validation sidebar in the editor now. The key thing for users with DevTools turned off is to identify the theme/plugins causing the problems, and not the problems themselves: highlight the cause not the symptoms.