Results from WPCLI that allows implement in w.org

[davidperezgar]

We need to give a result from WPCLI that fits meta team in order to execute this plugin in form submission.

This results fits the need asked by Meta #441 (comment), so we could implement as well as this #478 , and it would be the minimal needs to achieve that.

The results has a json format like this: {"pass":true,"errors":[],"warnings":[]}

The Pass value is true if it has no errors.

I'd need some feedback if the approach is correct, or we could do something more.

[joemcgill]

As asked in https://github.com/WordPress/plugin-check/issues/441#issuecomment-2195224108

I'm curious if the presence of any errors returned by the CLI command would be sufficient as a pass/fail signal, or if the command needs to also return a specific error code as well?

[davidperezgar]

As asked in #441 (comment)

I'm curious if the presence of any errors returned by the CLI command would be sufficient as a pass/fail signal, or if the command needs to also return a specific error code as well?

I think is the easiest way. We have to be sure that errors haven't got any false positive.

[github-actions[bot]]

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: davidperezgar <[email protected]>
Co-authored-by: mukeshpanchal27 <[email protected]>
Co-authored-by: ernilambar <[email protected]>
Co-authored-by: dd32 <[email protected]>
Co-authored-by: swissspidy <[email protected]>
Co-authored-by: joemcgill <[email protected]>
Co-authored-by: felixarntz <[email protected]>
Co-authored-by: barrykooij <[email protected]>
Co-authored-by: bordoni <[email protected]>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

[davidperezgar]

Thanks for your suggestions!

[joemcgill]

I don't particularly like the approach of creating a new output format specific for wporg if we are instead able to consume one of the exiting formats when passing the correct arguments to the CLI script.

Before trying to implement a solution here, can we get an answer from systems (e.g., @dd32) to the question asked in https://github.com/WordPress/plugin-check/issues/441#issuecomment-2195224108? We also need to decide if #479 is a blocker to this issue, because we may not need a special CLI command at all if we can configure what errors are returned based on severity levels.

[davidperezgar]

Related with #479 and #440 .

[mukeshpanchal27]

@davidperezgar I have fixed PHPMD issues and added behat test for new argument.

[ernilambar]

I tried this command:

wp plugin check foo-single.php --fields=line,column,code --format=wporg

Argument --fields is not respected when using wporg format.

[davidperezgar]

Yes, and it's correct. The format for wporg is very strict and I shouldn't be customized.

[dd32]

I don't see anything "wrong" here, but I don't see the purpose of it, other than to fix the fact that --format=json isn't a pure JSON output.

Instead of adding this for the purposes of adding it to w.org plugin submission, I'd probably just suggest fixing the existing json output, or request that we parse the existing json output.

My main issue with this PR is that:

'pass' => empty( $errors ) ? true : false,

Is completely useless. Pretty much every plugin throws up errors in plugin-check that are considered errors that aren't actually errors.

The original "A command that can be run, parsed, and provides a clear block or accept result" really still stands, and while this does do that, it still doesn't provide a the last part in any form that's inline with the plugin submission guidelines IMHO.

For example; Are this really something that should block a plugin submission?

| 0 | 0 | ERROR | outdated_tested_upto_header | Tested up to: 6.5 < 6.6 | | 37 | 25 | ERROR | WordPress.WP.I18n.MissingTranslatorsComment | A function call to __() with texts containing placeholders was found, but was not accompanied by a "translators:" comment on the line above to clarify the meaning of the placeholders. | | 59 | 21 | ERROR | WordPress.Security.EscapeOutput.ExceptionNotEscaped | All output should be run through an escaping function (see the Security sections in the WordPress Developer Handbooks), found '__'. | | 114 | 30 | ERROR | WordPress.WP.AlternativeFunctions.json_encode_json_encode | json_encode() is discouraged. Use wp_json_encode() instead. |

So I guess actually, this all comes back to #479 instead as the main blocker.

[davidperezgar]

Yes, We're working on severity levels, and the team where agreed equals and more than 7, should be an error.

So, I see:

• We should have a format=wporg as json is for users.
• We have we give you then after the severity level an ERROR or WARNING
• Do you need then the pass variable?

For the example that you give:

1. ERROR
2. WARNING
3. ERROR
4. ERROR

So we have to work on severity levels and then give you the result.

[davidperezgar]

Hello, I've updated the code with the severity level and wporg format. I've made within iterative for all formats, but creating an array for errors and warning after severity filter.

I've removed pass as is useless.

We can discuss what's your opinion.

[davidperezgar]

Thanks Nilambar for the changes, I think is ready for a review from @dd32

[dd32]

Am I understanding this correctly?

$output = json_decode( wp plugin-check .... )
$error_types = wp_list_pluck( $output->errors, 'type' );
$pass = ! in_array( 'ERROR', $error_types );

If so, yep, that's fine.

[davidperezgar]

Yes Dion! ERRORS will block an upload. But please show as well warnings as they will help to reduce the time for reviewing.

[davidperezgar]

@dd32 Do you need something else about this PR? Thanks.

[davidperezgar]

Ok!

[ernilambar]

Probably we dont need this PR now. It seems it is possible to integrate in org without new format.

[dd32]

Closing, we've taken a different direction in implementation. See https://meta.trac.wordpress.org/ticket/7778

[davidperezgar]

Ok! No problem, if the result is the same! ;)