icu icon indicating copy to clipboard operation
icu copied to clipboard

Make ICU ptest output compatible with Automake format

Open jiajia123-wind opened this issue 1 month ago • 7 comments

Update ICU ptest output so that each test prints in the Automake "simple test" format required by Yocto[1]:

result: testname

Where result is one of PASS, FAIL, or SKIP.

[1] https://wiki.yoctoproject.org/wiki/Ptest

Note: I do not currently have permissions to create an ICU Jira issue. I am submitting this patch to discuss the change and its approach with the ICU team. Once the PR is reviewed, a Jira issue can be created and associated if required.

jiajia123-wind avatar Nov 05 '25 05:11 jiajia123-wind

We don't want to unconditionally change the test suite output after 25+ years. A different format under a command line flag might be ok. We would also want a reference to the output format spec or what tool expects that.

What is "ptest"?

I do not currently have permissions to create an ICU Jira issue.

Please follow the instructions from https://icu.unicode.org/bugs

markusicu avatar Nov 06 '25 17:11 markusicu

Notice: the branch changed across the force-push!

  • icu4c/source/configure.ac is now changed in the branch
  • icu4c/source/icudefs.mk.in is now changed in the branch
  • icu4c/source/test/intltest/intltest.cpp is different
  • icu4c/source/tools/ctestfw/ctest.c is different

View Diff Across Force-Push

~ Your Friendly Jira-GitHub PR Checker Bot

@markusicu Thank you for the feedback!

You're right — changing the default test output would not be ideal after so many years. I’ve updated this patch to make the Automake-style output optional, controlled by the --enable-ptest-automake-format configure flag, instead of changing the default behavior.

Regarding your question:

  • ptest is the Yocto Project’s package testing framework.
    It expects each test case to print results in the Automake "simple test" format (PASS: testname, FAIL: testname, or SKIP: testname). Reference: https://wiki.yoctoproject.org/wiki/Ptest

Thanks for the clarification and the link!

I’ve followed the instructions on https://icu.unicode.org/bugs and successfully logged into the ICU JIRA site.
However, it seems my account ([email protected]) currently does not have permission to create new issues — the “Create” button is disabled for me.

Thanks again for the helpful comments!

jiajia123-wind avatar Nov 12 '25 08:11 jiajia123-wind

Notice: the branch changed across the force-push!

  • icu4c/source/configure.ac is different
  • icu4c/source/icudefs.mk.in is no longer changed in the branch

View Diff Across Force-Push

~ Your Friendly Jira-GitHub PR Checker Bot

Notice: the branch changed across the force-push!

  • icu4c/source/configure.ac is different
  • icu4c/source/test/intltest/intltest.cpp is different
  • icu4c/source/tools/ctestfw/ctest.c is different

View Diff Across Force-Push

~ Your Friendly Jira-GitHub PR Checker Bot

Notice: the branch changed across the force-push!

  • icu4c/source/configure.ac is different
  • icu4c/source/test/intltest/intltest.cpp is different
  • icu4c/source/tools/ctestfw/ctest.c is different

View Diff Across Force-Push

~ Your Friendly Jira-GitHub PR Checker Bot

Hi @thesamesam,

Thanks for your suggestion! I've updated the name in the configuration to reflect the more general term. The option is now named --enable-automake-test-format instead of --enable-ptest-automake-format, and the description has also been updated accordingly. This should make it clearer and more broadly applicable, beyond just Yocto.

jiajia123-wind avatar Nov 21 '25 05:11 jiajia123-wind