Test Explorer showing "red bang" icon instead of test status - Cannot use ruby code in project.yml

[Shayamir]

I've configured Ceedling to run using the plugin. I've also added xml_tests_report to my project.yml. For some reason, after running tests, I get a "red bang" icon, instead of a test status.

My Ceedling version: $ ceedling version Ceedling:: 0.31.1 Unity:: 2.5.4 CMock:: 2.5.4 CException:: 1.3.3 report.xml.txt

Any ideas how I can debug this further?

[numaru]

You should be able to see the output of the ceedling command in the OUTPUT panel by clicking on the test. Can you share this message?

The report.xml seems legit to me, no error in it, only one failure.

[Shayamir]

It looks like this: stdout:

Test 'test_bar.c'

Running test_bar.out...

---

TEST OUTPUT

[test_bar.c]

• "Processing message for device of type 3, index 10"
• "Processing message for device of type 3, index 10"
• "Processing message for device of type 3, index 10"
• "Processing message for device of type 3, index 10"
• "Processing message for device of type 3, index 10"
• "Processing message for device of type 3, index 10"
• "Processing message for device of type 3, index 10"
• "Processing message for device of type 3, index 10"
• "Processing message for device of type 3, index 14"
• "Processing message for device of type 3, index 14"

---

FAILED TEST SUMMARY

[test_bar.c] Test: test_cs_access_write_lan_bar0_msb At line (176): "Expected 1 Was 0"

---

OVERALL TEST SUMMARY

TESTED: 5 PASSED: 4 FAILED: 1 IGNORED: 0

[Shayamir]

BTW, I've made 1 test fail, to check the different status icons. BTW2, I was sure that the results are being parsed back from XML report and that the OUPUT is not so relevant... Probably I was mistaken.

[Shayamir]

Any idea how I can debug? Can you help?

[Shayamir]

I've debugged the issue. My project.yml (relevant sections) looks like this:

:project:
  :build_root: "#{ENV['BUILD_ROOT']}"
  :test_file_prefix: test_
  :default_tasks:
    - test:all

:environment:
  - REPO_ROOT: "#{ENV.fetch('REPO_ROOT', ENV['PWD'])}"
  - BUILD_ROOT: "#{ENV.fetch('BUILD_ROOT', ENV['PWD'] + '/build/unittest_ceedling/debug')}"

:paths:
  :test:
    - "#{ENV['REPO_ROOT']}/controller/**/test"
  :source:
    - "#{ENV['REPO_ROOT']}/controller/**"

When the extension runs, in file: adapter.ts, function: getXmlReportPath, I see this path: 'C:\prj\mmg\#{ENV['BUILD_ROOT']}\artifacts\test\report.xml'

It seems like Environment variable is not being expanded.

any idea how to proceed?

[Shayamir]

Narrowed it down to this line: const ymlProjectBuildDirectory = ymlProjectData[':project'][':build_root']; returns: ymlProjectBuildDirectory = '#{ENV['BUILD_ROOT']}'

[numaru]

Sorry for the late reply and thank you for taking the time to debug the issue. Do you use ruby code elsewhere in the project.yml? Do you really need those environment variables?

How I would implement it:

1st option

• When the paths are get: (cached)
  • parse the environment section of the project.yml
  • if a path to ruby is defined in the extension config or in the system path then:
    • exec the path prepend by the definition of the env variables as ruby code with child_process.exec('ruby', ...code) and return the resulting string
  • else:
    • return the raw string

2nd option

• When the paths are get: (cached)
  • run ceedling paths:* to get all the paths searched by ceedling
  • parse the result to get :source and :test

[Shayamir]

3rd Option I'd write a routine in typescript files to read from .yml and if It finds '#{ENV[' string, it'll replace it with the corresponding environment variable.

4th Option Don't fix this, and document which variables on project.yml cannot use environment variables.

Since Ceedling .yml files support environment variables, other people might encounter this problem.

Right now, for me, I've decided to not use the environment variable in :project:build_root: And the extension is working fine. BTW, thank you very much for a great extension.

[numaru]

Great! I will let it as it is for now and keep the issue open.

[breakersun]

Got the same problem here: ceedling versions(on Windows 10) ❯ ceedling version
Ceedling:: 0.32.0 Unity:: 2.5.4 CMock:: 2.5.4 CException:: 1.3.3

Project

---
:project:
  # how to use ceedling. If you're not sure, leave this as `gem` and `?`
  :which_ceedling: gem
  :ceedling_version: 0.32.0

  # optional features. If you don't need them, keep them turned off for performance
  :use_mocks: TRUE
  :use_test_preprocessor: TRUE
  :use_preprocessor_directives: FALSE
  :use_deep_dependencies: FALSE
  :use_auxiliary_dependencies: TRUE
  :use_backtrace_gdb_reporter: FALSE

  # tweak the way ceedling handles automatic tasks
  :build_root: build
  :test_file_prefix: test_
  :default_tasks:
    - test:all

  # performance options. If your tools start giving mysterious errors, consider 
  # dropping this to 1 to force single-tasking
  :test_threads: 8
  :compile_threads: 8

  # you can specify different yaml config files which modify the existing one
  :options_paths: []

  # enable release build (more details in release_build section below)
  :release_build: FALSE

# specify additional yaml files to automatically load. This is helpful if you
# want to create project files which specify your tools, and then include those
# shared tool files into each project-specific project.yml file.
:import: []

# further details to configure the way Ceedling handles test code
:test_build:
  :use_assembly: FALSE
  :toolchain_include: []

# further details to configure the way Ceedling handles release code
:release_build:
  :output: MyApp.out
  :use_assembly: FALSE
  :artifacts: []
  :toolchain_include: []

# Plugins are optional Ceedling features which can be enabled. Ceedling supports
# a variety of plugins which may effect the way things are compiled, reported, 
# or may provide new command options. Refer to the readme in each plugin for 
# details on how to use it.
:plugins:
  :load_paths: []
  :enabled:
    - module_generator               # handy for quickly creating source, header, and test templates
    - stdout_pretty_tests_report
    - xml_tests_report

# override the default extensions for your system and toolchain
:extension:
  :executable: .out

# This is where Ceedling should look for your source and test files.
# see documentation for the many options for specifying this.
:paths:
  :test:
    - +:test/**
    - -:test/support
  :source:
    - src/**
  :support:
    - test/support
  :libraries: []

# Configuration Options specific to CMock. See CMock docs for details
:cmock:
  :mock_prefix: mock_
  :when_no_prototypes: :warn
  :enforce_strict_ordering: TRUE
  :plugins:
    - :ignore
    - :ignore_arg
    - :expect_any_args
    - :array
    - :callback
    - :return_thru_ptr
  :treat_as:
    uint8:    HEX8
    uint16:   HEX16
    uint32:   UINT32
    int8:     INT8
    bool:     UINT8

# Configuration options specific to Unity. 
:unity:
  :defines:
    - UNITY_EXCLUDE_FLOAT

# You can optionally have ceedling create environment variables for you before
# performing the rest of its tasks.
:environment: []

with all tests passed, still get a red bang symbol: