Summary of Test Execution Does Not Match Bazel Output

[danielburn5]

Description of the bug:

Running a bazel test command via an IntelliJ run config produces what looks like a successfully executed test run, with all tests passing. However the bazel console output shows a failure.

The screenshot below (sensitive details removed) shows the summary indicating 80 of 80 tests passed, whilst the console output shows only 1 test was executed, which failed.

The run config looks like this when exported (with anything sensitive removed and replaced with foobar placeholders):

<configuration name="Bazel test Foo" type="BlazeCommandRunConfigurationType" factoryName="Bazel Command"
               temporary="true">
    <blaze-settings handler-id="BlazeJavaRunConfigurationHandlerProvider" kind="java_test"
                    context-element="$PROJECT_DIR$/../foo/bar/src/test/java/net/Foo.java#myUnitTest"
                    blaze-command="test">
        <blaze-target>
            //foo/bar:src/test/java/net/Foo
        </blaze-target>
        <blaze-user-flag>--test_sharding_strategy=disabled</blaze-user-flag>
        <env_state>
            <envs>
                <env name="FOO" value="foo"/>
                <env name="BAR" value="bar"/>
            </envs>
        </env_state>
    </blaze-settings>
    <method v="2">
        <option name="Blaze.BeforeRunTask" enabled="true"/>
    </method>
</configuration>

Which category does this issue belong to?

IntelliJ

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Executing the run config when a test failure occurs. However, this does not happen everytime, so it is difficult to reproduce.

Which Intellij IDE are you using? Please provide the specific version.

Working on a remote project via coder, as such using the JetBrains client:

JetBrains Client 2023.3
Build #JBC-233.11799.241, built on December 2, 2023
Runtime version: 17.0.9+7-b1087.7 x86_64
VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o.
macOS 14.4
Controller in Remote Development (IU), direct control type: Lux
GC: G1 Young Generation, G1 Old Generation
Memory: 2048M
Cores: 16
Metal Rendering is ON
Registry:
  editor.minimap.enabled=true
  rdclient.backend.actionHandlers=false
Non-Bundled Plugins:
  com.intellij.plugins.eclipsekeymap (233.11799.165)
  ru.adelf.idea.dotenv (2024.1)
  com.tylerthrailkill.intellij.solarized (3.0.0)
  com.intellij.plugins.visualstudiokeymap (233.11799.165)
  com.intellij.plugins.netbeanskeymap (233.11799.165)

What programming languages and tools are you using? Please provide specific versions.

Java

What Bazel plugin version are you using?

bundled 2024.03.12.0.1-api-version-233

Have you found anything relevant by searching the web?

No

Any other information, logs, or outputs that you want to share?

No

[blorente]

Hi! Sorry you're seeing the issue. I've tried to reproduce in one of our example projects (examples/java/greetings_project), and it doesn't reproduce (ran it 10 times, it showed as failed all 10 times).

Could you provide with an example test where the error reproduces? I suspect this is an issue with the XML JUnit report that's being emitted by the test and read by the issue parser, so having the specific test that fails would help understand the problem.

[github-actions[bot]]

Thank you for contributing to the IntelliJ repository! This issue has been marked as stale since it has not had any activity in the last 6 months. It will be closed in the next 14 days unless any other activity occurs. If you think this issue is still relevant and should stay open, please post any comment here and the issue will no longer be marked as stale.

[github-actions[bot]]

This issue has been automatically closed due to inactivity. If you're still interested in pursuing this, please post @bazelbuild/triage in a comment here and we'll take a look. Thanks!