aqa-test-tools icon indicating copy to clipboard operation
aqa-test-tools copied to clipboard

Test GlitchWitcher Workflow

Open anirudhsengar opened this issue 3 months ago • 6 comments

We want to check whether the GlitchWitcher workflow runs as intended against PR's with C/C++ files from different repo's. This is to find any errors that may have been missed during development. The PAT needs to be verified as it is an organization level PAT not repo level.

Test cases -

  1. PR with C/C++ files only from OpenJ9/OpenJDK as the dataset is already available in adoptium/aqa-triage-data
  2. PR with files from different languages including C/C++ files from OpenJ9/OpenJDK
  3. PR with C/C++ files from repos where the analysis hasn't been performed before (Will trigger the use of PAT)
  4. PR with files from different languages including C/C++ files from new repos
  5. PR with no C/C++ files at all

anirudhsengar avatar Aug 29 '25 05:08 anirudhsengar

GlitchWitcher https://github.com/eclipse-openj9/openj9/pull/22531

anirudhsengar avatar Aug 29 '25 05:08 anirudhsengar

🔮 GlitchWitcher Analysis Results

Target PR: https://github.com/eclipse-openj9/openj9/pull/22531 Repository: eclipse-openj9-openj9

📊 Bug Prediction Analysis

File: InlinerTempForJ9.cpp

Outcome: Defective -> Defective

Metric BEFORE PR AFTER PR % Change
PDF(Defective | Reconstruction Error) 5.166175438419213e-11 5.157659826374885e-11 -0.16%
PDF(Non-Defective | Reconstruction Error) 9.136527266129927e-13 9.117070032916298e-13 -0.21%

📋 Interpretation Note:

The values shown are Probability Densities (PDFs), not probabilities. They represent the model's assessment of how likely a file's characteristics are to be 'defective' vs. 'non-defective'. A higher value indicates a better fit for that category. Very small values are expected and normal.

Analysis performed by GlitchWitcher Bot

github-actions[bot] avatar Aug 29 '25 05:08 github-actions[bot]

GlitchWitcher https://github.com/openjdk/jdk/pull/26997

anirudhsengar avatar Aug 29 '25 05:08 anirudhsengar

🔮 GlitchWitcher Analysis Results

Target PR: https://github.com/openjdk/jdk/pull/26997 Repository: openjdk-jdk

📊 Bug Prediction Analysis

File: asmtest.out.h

Outcome: Defective -> Defective

Metric BEFORE PR AFTER PR % Change
PDF(Defective | Reconstruction Error) 5.612720036520575e-10 3.808528736177021e-10 -32.14%
PDF(Non-Defective | Reconstruction Error) 2.063202565568776e-10 1.4820650730103194e-10 -28.17%

File: assembler_x86.hpp

Outcome: Defective -> Defective

Metric BEFORE PR AFTER PR % Change
PDF(Defective | Reconstruction Error) 4.3274766367346293e-10 4.270703428262017e-10 -1.31%
PDF(Non-Defective | Reconstruction Error) 1.6518968429904368e-10 1.6334374454098219e-10 -1.12%

File: assembler_x86.cpp

Outcome: Non-Defective -> Non-Defective

Metric BEFORE PR AFTER PR % Change
PDF(Defective | Reconstruction Error) 6.516866505691497e-13 6.5345328999376e-13 +0.27%
PDF(Non-Defective | Reconstruction Error) 1.08870956056049e-12 1.0907975081553753e-12 +0.19%

📋 Interpretation Note:

The values shown are Probability Densities (PDFs), not probabilities. They represent the model's assessment of how likely a file's characteristics are to be 'defective' vs. 'non-defective'. A higher value indicates a better fit for that category. Very small values are expected and normal.

Analysis performed by GlitchWitcher Bot

github-actions[bot] avatar Aug 29 '25 06:08 github-actions[bot]

GlitchWitcher https://github.com/eclipse-openj9/openj9/pull/22660

anirudhsengar avatar Sep 24 '25 09:09 anirudhsengar

🔮 GlitchWitcher Analysis Results

Target PR: https://github.com/eclipse-openj9/openj9/pull/22660 Repository: eclipse-openj9-openj9

📊 Bug Prediction Analysis

File: JFRConstantPoolTypes.hpp

Outcome: Defective -> Defective

Metric BEFORE PR AFTER PR % Change
PDF(Defective | Reconstruction Error) 1.6749436697891118e-08 1.0667636346537195e-07 +536.90%
PDF(Non-Defective | Reconstruction Error) 1.875894025468385e-09 2.373655990255831e-08 +1165.35%

File: VMHelpers.hpp

Outcome: Defective -> Defective

Metric BEFORE PR AFTER PR % Change
PDF(Defective | Reconstruction Error) 1.7347438426804454e-08 9.345989554629563e-08 +438.75%
PDF(Non-Defective | Reconstruction Error) 1.967179036237612e-09 1.9755329332244272e-08 +904.25%

File: JFRChunkWriter.cpp

Outcome: Defective -> Defective

Metric BEFORE PR AFTER PR % Change
PDF(Defective | Reconstruction Error) 5.047891705443575e-08 1.6184928883544795e-08 -67.94%
PDF(Non-Defective | Reconstruction Error) 8.443450445013651e-09 1.7908088381930018e-09 -78.79%

File: JFRChunkWriter.hpp

Outcome: Defective -> Defective

Metric BEFORE PR AFTER PR % Change
PDF(Defective | Reconstruction Error) 1.1049194155323313e-07 4.688212496914575e-08 -57.57%
PDF(Non-Defective | Reconstruction Error) 2.492481178497733e-08 7.62859330217048e-09 -69.39%

📋 Interpretation Note:

The values shown are Probability Densities (PDFs), not probabilities. They represent the model's assessment of how likely a file's characteristics are to be 'defective' vs. 'non-defective'. A higher value indicates a better fit for that category. Very small values are expected and normal.

Analysis performed by GlitchWitcher Bot

github-actions[bot] avatar Sep 24 '25 09:09 github-actions[bot]