ACS 3.0 some of jason is showing no waive

[quic-pansing]

Picture attached. As per guideline BSA failure will not impact on IR certification. but ACS 3.0 jaison mention that no waiver.

[chetan-rathore]

Hi @quic-pansing,

There are some changes done for overall compliance result on top of last BETA release. We are planning next release around April last week.

Can you try with the latest DT image: https://github.com/chetan-rathore/arm-systemready/tree/3.0_rc_images/rc-1/24thmar and let us know if the issue mentioned is still observed.

In case if you need to build latest image at your end, please use below branch https://github.com/ARM-software/arm-systemready/tree/3.0_development

Thanks, Chetan

[quic-pansing]

@chetan-rathore we checked with latest image, here is below issue we found

Second issue is: when apply waiver, on logs is showing waiver applied but Dashboard there is no waiver shown

[chetan-rathore]

Hello @quic-pansing,

Thanks for running the latest image and sharing the feedbacks.

1. SCT is a mandatory test suite for overall compliance, and not waivable. So waivers will not be applied on SCT.
2. Do you mean acs_summary**.xml** or acs_summary**.json**, as currently support for XML is not provided and JSON is the standard format.
3. BSA is recommended for DT compliance and hence not impact overall compliance.
4. BBSR is also recommended for now.

Thanks, Chetan

[quic-pansing]

@chetan-rathore SCT Is mandatory but if some particular test case is failed due to Limitation or other reason, how waiver can apply?

[chetan-rathore]

@quic-pansing ..can you share more details on which sct test failed and the limitation...you can send email to [email protected] with details

[quic-pansing]

sure. Will share over email..

[chetan-rathore]

Hi @quic-pansing,

Thanks for sharing the results, we are discussing internally the results.

[quic-pansing]

@chetan-rathore any update for above. When we can expect answer

[chetan-rathore]

@chetan-rathore any update for above. When we can expect answer

Hi @quic-pansing,

We are discussing internally and will share response once have the updates.

Thanks, Chetan

[quic-pansing]

Hey @chetan-rathore, did you get any response?

[chetan-rathore]

Hi @quic-pansing,

I have shared the response and the systemready compliance FAQ guide which will help to understand the method of declaration of compliance after pivot to self guided compliance.

Thanks, Chetan

[ownia]

Hi @chetan-rathore, could you share the FAQ guide with me? Some of partners may need this.

[chetan-rathore]

Hi @ownia,

Please find the guide link: https://developer.arm.com/documentation/107981/0303/SystemReady-Frequently-Asked-Questions

Thanks, Chetan

[ownia]

Thanks @chetan-rathore. I noticed that it contains a "SystemReady band – General FAQ" section, do we have plans to add a Devicetree Band section?

[chetan-rathore]

Hi @ownia,

I also noticed it, but please note as pivot from certification to self guided compliance is for both bands most of non ES/SR information is valid for both bands.

But I will sync internally to have FAQ guide more clearer.

[quic-pansing]

Hi @quic-pansing,

I have shared the response and the systemready compliance FAQ guide which will help to understand the method of declaration of compliance after pivot to self guided compliance.

Thanks, Chetan

Network protocol related failure PlatformSpecificElements_0_0_A0A8BED3-3D6F-4AD8-907A-84D52EE1543B.log

Query mode related failure logs QueryMode_Func_0_0_D493E3FD-6B56-4F8E-BA45-606D0E8F5671.log

[chetan-rathore]

Hi @quic-pansing,

Please find the responses from our side.

• UEFI Compliant failure on HII: This is a false positive due to recent U-Boot now implementing HiiConfigRouting and the SCT parser config not being up to date. This can be ignored.

• UEFI Compliant failure on network boot: This is a true failure: EFI_SIMPLE_NETWORK_PROTOCOL is missing (SNP - No).

• QueryMode failure on mode 0: This looks like a true failure, is this a special setup with console on both UART & GOP maybe? Can you share details of your setup.

Thanks, Chetan

[quic-pansing]

Thanks @chetan-rathore Yes we understand this failure is true but we have asked that how we can apply for waiver?

For the device tree band, network boot is not a mandatory requirement. Additionally, if device does not support network boot, so protocol enablement is not necessary.

Yes, this true. we are using both protocol UART and GOP. for this also we are looking for Waiver.

[chetan-rathore]

Hi @quic-pansing ,

I will have some updates on your latest query by end of next week.

Thanks, Chetan

[psingh3086]

@chetan-rathore did you get any update?

[chetan-rathore]

Hi @psingh3086,

As the SystemReady program has pivoted to a self-guided compliance model, partners are now expected to review the root cause of the failure and document any applicable waivers internally. The ACS image will not auto-apply the waivers for failures in mandatory test cases.

For the failures related to ACS suite, I will be sharing a SystemReady DT intermittent image, which includes fixes for known false failures due to tooling or test issues within the ACS. This should help ensure accurate results and reduce noise in your compliance testing.

Thanks, Chetan

[chetan-rathore]

Hi @psingh3086,

Can you run with this latest debug image and share the observations.

https://github.com/chetan-rathore/arm-systemready/tree/v3.0.2/SystemReady-devicetree-band/prebuilt_images/v3.0.1plus-rc0

If the acs_results folder can be shared, it will be great help.

Thanks, Chetan

[smangali-qc]

HI @chetan-rathore ,

Are you still looking for results on the above debug build ? please let me know

when are you planning to release prebuilt binary with latest changes ?

Regards, Shiva Kumar

[chetan-rathore]

Hello @smangali-qc,

Yes, I need confirmation that the BSA run is successful and no hang/exception observed with the above shared binary. Based on that I will share a new debug DT image. Plan is to share the DT image by coming week, by 12th or earlier.

Thanks, Chetan

[smangali-qc]

HI @chetan-rathore ,

bsa_test.log

we have observed hang while running bsa test at 502 : Wake from EL1 VIR Timer Int .

regards, shiva kumar

[chetan-rathore]

Hi @smangali-qc,

Please use this DT image, this BSA issue has been fixed. https://github.com/chetan-rathore/arm-systemready/tree/Images/SystemReady-devicetree-band/prebuilt_images/v3.1.0-rc0-12Sep

[smangali-qc]

HI @chetan-rathore , we are still observing same failure with the above image. getting stuck 501 test

logs: 407 : Check System Counter Frequency
START Using F/W interrupt API's

   Counter frequency is 19 MHz
   B_TIME_02
                                   : Result:  PASS
   END


  One or more Timer tests failed or were skipped.

  *** Starting Wakeup semantic tests ***

Operating System View: 501 : Wake from EL1 PHY Timer Int
START Using F/W interrupt API's Using F/W interrupt API's Using F/W interrupt API's

   PE wakeup by some other event

Regards, shiva kumar

[chetan-rathore]

Thanks for quick update, will have a look

[chetan-rathore]

Hi @smangali-qc,

The 501 test getting stuck is bit odd, based on prints shared. After the "PE wakeup by some event", the test sets SKIP status and returns back. https://github.com/ARM-software/sysarch-acs/blob/c86edea4ce28ed0c49cd5cf854809b6a783cfa79/test_pool/power_wakeup/u001.c#L82

Can you share the complete logs with higher verbosity.

Bsa.efi -v 1

Thanks, Chetan

[chetan-rathore]

Hi @smangali-qc,

Any update on the higher verbosity logs.