ceedling 1.0.0 release candidate

[mvandervoord]

I'm going to start building the 0.32 release candidate here. It might be a bit early for a PR, but this makes it easier to test on Actions.

Comments, help, and thoughts are welcome.

[swaldhoer]

@mvandervoord will you actively ask for feedback before merging this/creating a release?

I would be interested to provide feedback once this pull request reaches a good stability from your point of view.

[mvandervoord]

My purpose of labeling this branching so obviously was to solicit feedback. Let it roll in anytime :)

[deltalejo]

Hi! This version will include the improved support for separate out directories?

[swaldhoer]

My purpose of labeling this branching so obviously was to solicit feedback. Let it roll in anytime :)

I updated to c98953e89c0047b6716565a6dcb1a7f354d1df6a of this branch and see some strange behavior:

• The first clean run always fails

  Collecting Definitions
  ------------------------
  
  Getting Includes From Test Files
  --------------------------------
  ERROR: Ceedling Failed
  
  ERROR: Shell command failed.
  > Shell executed command:
  ...
  > And exited with status: [pid 14668 exit 1].
  
  #<Thread:0x000001cecf40bd30 C:/dummy/tools/vendor/ceedling/lib/ceedling/par_map.rb:7 run> terminated with exception (report_on_exception is true):
  C:/dummy/tools/vendor/ceedling/lib/ceedling/tool_executor.rb:84:in `exec': ShellExecutionException (ShellExecutionException)
  from C:/dummy/tools/vendor/ceedling/lib/ceedling/preprocessinator_includes_handler.rb:49:in `form_shallow_dependencies_rule'
  from C:/dummy/tools/vendor/ceedling/lib/ceedling/preprocessinator_includes_handler.rb:93:in `extract_includes_helper'
  from C:/dummy/tools/vendor/ceedling/lib/ceedling/preprocessinator_includes_handler.rb:77:in `extract_includes'
  from C:/dummy/tools/vendor/ceedling/lib/ceedling/preprocessinator.rb:49:in `preprocess_shallow_includes'
  from C:/dummy/tools/vendor/ceedling/lib/ceedling/preprocessinator.rb:17:in `preprocess_test_file'
  from C:/dummy/tools/vendor/ceedling/lib/ceedling/test_invoker.rb:114:in `block (2 levels) in setup_and_invoke'
  from C:/dummy/tools/vendor/ceedling/lib/ceedling/par_map.rb:10:in `block (2 levels) in par_map'
  rake aborted!
  ShellExecutionException: ShellExecutionException
  C:/dummy/tools/vendor/ceedling/lib/ceedling/tool_executor.rb:84:in `exec'
  C:/dummy/tools/vendor/ceedling/lib/ceedling/preprocessinator_includes_handler.rb:49:in `form_shallow_dependencies_rule'
  C:/dummy/tools/vendor/ceedling/lib/ceedling/preprocessinator_includes_handler.rb:93:in `extract_includes_helper'
  C:/dummy/tools/vendor/ceedling/lib/ceedling/preprocessinator_includes_handler.rb:77:in `extract_includes'
  C:/dummy/tools/vendor/ceedling/lib/ceedling/preprocessinator.rb:49:in `preprocess_shallow_includes'
  C:/dummy/tools/vendor/ceedling/lib/ceedling/preprocessinator.rb:17:in `preprocess_test_file'
  C:/dummy/tools/vendor/ceedling/lib/ceedling/test_invoker.rb:114:in `block (2 levels) in setup_and_invoke'
  C:/dummy/tools/vendor/ceedling/lib/ceedling/par_map.rb:10:in `block (2 levels) in par_map'
  Tasks: TOP => test:all
  (See full trace by running task with --trace)
  

• And the second goes further, but then exists with with missing the header:

  ...
  > Produced output:
  In file included from ../../src/app/main/include/general.h:59,
  from ../../src/app/application/config/battery_system_cfg.h:64,
  from ../../src/app/engine/config/database_cfg.h:62,
  from ../../src/app/engine/database/database.h:61:
   ../../src/app/main/include/fassert.h:249:10: fatal error: CException.h: No such file or directory
   #include "CException.h"
   ^~~~~~~~~~~~~~
   compilation terminated.
   > And exited with status: [pid 11696 exit 1].
  
  #<Thread:0x000001da13988110 C:/dummy/tools/vendor/ceedling/lib/ceedling/par_map.rb:7 run> terminated with exception (report_on_exception is true):
  C:/dummy/tools/vendor/ceedling/lib/ceedling/tool_executor.rb:84:in `exec': ShellExecutionException (ShellExecutionException)
  from C:/dummy/tools/vendor/ceedling/lib/ceedling/preprocessinator_file_handler.rb:12:in `preprocess_file'
  from C:/dummy/tools/vendor/ceedling/lib/ceedling/preprocessinator.rb:65:in `preprocess_file'
  from C:/dummy/tools/vendor/ceedling/lib/ceedling/preprocessinator.rb:33:in `preprocess_mockable_header'
  from C:/dummy/tools/vendor/ceedling/lib/ceedling/test_invoker.rb:139:in `block (2 levels) in setup_and_invoke'
  from C:/dummy/tools/vendor/ceedling/lib/ceedling/par_map.rb:10:in `block (2 levels) in par_map'
  rake aborted!
  ShellExecutionException: ShellExecutionException
  C:/dummy/tools/vendor/ceedling/lib/ceedling/tool_executor.rb:84:in `exec'
  C:/dummy/tools/vendor/ceedling/lib/ceedling/preprocessinator_file_handler.rb:12:in `preprocess_file'
  C:/dummy/tools/vendor/ceedling/lib/ceedling/preprocessinator.rb:65:in `preprocess_file'
  C:/dummy/tools/vendor/ceedling/lib/ceedling/preprocessinator.rb:33:in `preprocess_mockable_header'
  C:/dummy/tools/vendor/ceedling/lib/ceedling/test_invoker.rb:139:in `block (2 levels) in setup_and_invoke'
  C:/dummy/tools/vendor/ceedling/lib/ceedling/par_map.rb:10:in `block (2 levels) in par_map'
  Tasks: TOP => test:all
  (See full trace by running task with --trace)
  

And this is strange, as I did not change the project structure, the project.yml, how we vendor ceedling in the repository. So, CException.h is exactly at the some directory location as it has worked with ceedling v0.31.1.

Any idea, what might introduce this?

[mvandervoord]

@swaldhoer -- thanks for doing some testing!

First, for future reference, when posting a block of code on a github discussion like this, you can put the block of code (or command prompt response, in this case) between lines that contain nothing but three backticks like "```". This will make a block that preserves all the whitespace, unlike the single backticks you were using above. :)

# ```
# like this, without the comments at the beginning
# ```

On to the real topics!

First, do I understand correctly that you originally created your project to install to the vendor folder? How did you install the latest test release? Did you just download and copy over the vendor directory? This is going to make a weird little mix where you're doing your command parsing with the old version of ceedling, but using the new version in the guts. It should mostly work, but might create some quirks.

This problem is my fault. I should create a gem for you to be able to test with. I'll do so soon and post it for people to try. Then, once the new gem is installed, you can do ceedling upgrade <dirname> from just outside the project folder to upgrade it.

Finally, once we have you running a full new version, if you run into problems, if you could use ceedling verbosity[5] clobber test:all for when you capture problems, it'd be good to see all the details.

Thanks again! I should have a gem posted soon.

[Letme]

@mvandervoord can you put the gem creation in the CI, so that we can simply download it here? Should be much easier and requires you not to push it anywhere.

[mvandervoord]

@Letme -- that's a great idea. I'll see if Github Actions has that option :)

[swaldhoer]

First, do I understand correctly that you originally created your project to install to the vendor folder? How did you install the latest test release? Did you just download and copy over the vendor directory? This is going to make a weird little mix where you're doing your command parsing with the old version of ceedling, but using the new version in the guts. It should mostly work, but might create some quirks.

I installed the required gems from the commit c98953e89c0047b6716565a6dcb1a7f354d1df6a. And then updated our ceedling copy. We wrap ceedling into our build tool, so that there is only one tool to run everything from an top level view. Basically our setup is like this:

• ./build/unit_test: this directory is created by the build tool, from there ceedling is run and is where the project.yml is located
• tools/vendor/ceedling/*: our ceedling copy

So we basically use the vendor approach, but some paths are tweaked to match our project setup. If you are interested I can further explain.

This problem is my fault. I should create a gem for you to be able to test with. I'll do so soon and post it for people to try. Then, once the new gem is installed, you can do ceedling upgrade <dirname> from just outside the project folder to upgrade it.

Finally, once we have you running a full new version, if you run into problems, if you could use ceedling verbosity[5] clobber test:all for when you capture problems, it'd be good to see all the details.

Thanks again! I should have a gem posted soon.

Ok, Then I will wait for a new gem and test again. I could overcome the include issue by adding ../../tools/vendor/ceedling/vendor/cmock/src etc. to the include paths. But with 0.31.1, there was no reason to add these include paths, and ceedling somehow magical knew where to search for these specific include paths. However, the build than ran for ever and I needed to cancel it.

I will only be able to run the next try on thursday (2023-02-16), but I will do for sure and post an update. We are very willing to keep our dependencies up to date and if this helps to make this release better, we are glad to help :)

[mvandervoord]

@swaldhoer -- There's a prerelease gem posted in releases: https://github.com/ThrowTheSwitch/Ceedling/releases/download/0.32.0-665b570/ceedling-0.32.0-665b570.gem

@Letme -- Thanks for the excellent idea. I'm liking automatic prereleases.

[swaldhoer]

@swaldhoer -- There's a prerelease gem posted in releases: https://github.com/ThrowTheSwitch/Ceedling/releases/download/0.32.0-665b570/ceedling-0.32.0-665b570.gem

@Letme -- Thanks for the excellent idea. I'm liking automatic prereleases.

Sorry this is just a memory dump from friday. I used the gem you linked and then updated the vendored files using the cli arguments.

• Starting from a clean repo state. (git clean -xdf) and then running ceedling, errors out, as ceedling calls the compiler and expects some file foo/_temp/abc.c and the file is not yet created.
• re-running the command again, ceedling creates 100% cpu usage, takes quite some time, and then misses some headers (the same mentioned above). By adding the ceedling vendored headers to the list of includes in the project.yml (e.g., ../../tools/vendor/ceedling/vendor/cmock/src), the headers are found.
• re-running ceedling again takes for ever (longer than with the current release) and then fails to compile the first(!) test case.

Edit: regarding the step 2: what has been changed in the way ceedling looks for unity etc. headers?

[lukzeg]

Hi @swaldhoer ,

Do you know maybe which header file is causing mentioned by you problem? As mostly I had similar issue in the past as you. It was connected with multiple of the pre-processor statements.

Finding this file and sharing project.yml will help to investigate what change cause this issue.

[swaldhoer]

Hi @swaldhoer ,

Do you know maybe which header file is causing mentioned by you problem? As mostly I had similar issue in the past as you. It was connected with multiple of the pre-processor statements.

Finding this file and sharing project.yml will help to investigate what change cause this issue.

Unfortunately the answer is simple: all header files that are vendores by ceedling.

I will share the project yaml later/tomorrow.

[swaldhoer]

Here is the project configuration. Imporant:

• the project files is in path/to/repo/build/unit_test
• the vendored ceedling is in path/to/repo/tools/vendor/ceedling. To avoid any ambiguity about the vendoring: the ceedling binary is then at path/to/repo/tools/vendor/ceedling/bin/ceedling

project.yml

:project:
  :use_exceptions: TRUE
  :use_test_preprocessor: TRUE
  :use_mocks: TRUE
  :use_auxiliary_dependencies: TRUE
  :build_root: .
  :test_file_prefix: test_
  :which_ceedling: ../../tools/vendor/ceedling
  :ceedling_version: 0.31.1
  :default_tasks:
    - test:all

:environment: []

:extension:
  :executable: .out

:paths:
  :test:
    - +:../../tests/unit/**
    - -:../../tests/unit/support
  :source:
    - ../../src/app/**
    - ../../src/opt/**
  :include:
    - +:./include/**
    - +:../../src/**
    - -:../../src/app/**
    - -:../../src/opt/**
  :support:
    - ../../tests/unit/support/**

:defines:
  :common: &common_defines
    - UNITY_INCLUDE_EXEC_TIME
  :test:
    - *common_defines
  :test_preprocess:
    - *common_defines
    - INC_FREERTOS_H

:cmock:
  :when_no_prototypes: :warn
  :enforce_strict_ordering: TRUE
  :mock_prefix: Mock
  :weak: ""
  :strippables:
    [
      "(.FREERTOS_SYSTEM_CALL)",
      "(.PRIVILEGED_FUNCTION)",
      "(portDONT_DISCARD)",
    ]
  :includes:
    - "FreeRTOSConfig.h"
    - "FreeRTOS.h"
    - "portmacro.h"
    - "mpu_wrappers.h"
    - "portable.h"
    - "task.h"
    - "queue.h"
    - "semphr.h"
    - "stream_buffer.h"
    - "event_groups.h"
    - "string.h"
  :plugins:
    - :ignore
    - :callback
    - :ignore_arg
    - :return_thru_ptr
  :treat_externs: :include
  :treat_as:
    uint8: HEX8
    uint16: HEX16
    uint32: UINT32
    int8: INT8
    bool: UINT8

:tools_test_linker:
  :arguments:
    - -lm
    - -flto
:tools_gcov_linker:
  :arguments:
    - -lm
    - -flto
:gcov:
  :utilities:
    - gcovr
  :reports:
    - HtmlDetailed
    - Text
    - Cobertura
  :abort_on_uncovered: true
  :gcovr:
    :report_root: "../../"
    :report_exclude: ".*vendor.*|.*build.*|.*test.*|.*tests.*|.*lib.*|.*Test.*|.*src.hal.*|.*src.os.*"
    :exclude_directories: ".*tests.*|.*src.os.*|.*build.*"
    :sort_percentage: true
    :sort_uncovered: false
    :html_medium_threshold: 60
    :html_high_threshold: 85
    :print_summary: true
    :num_parallel_threads: 4
    :keep: false

:junit_tests_report:
  :artifact_filename: report_junit.xml

:plugins:
  :load_paths:
    - ../../tools/vendor/ceedling/plugins
  :enabled:
    - gcov
    - stdout_pretty_tests_report
    - module_generator
    - xml_tests_report
    - junit_tests_report

[mvandervoord]

@swaldhoer -- it sounds like the new automatic generation of the gem isn't adding the submodules (unity, cmock, etc) to the gem. I'll figure out what's going on with that.

More concerning are the other failures. It's looking like something with the updated preprocessor calls isn't happy? Can you do a quick test where you disable the preprocessor and rake clobber test:all to see if it will complete a test then?

As a side note, feel free to add the following two lines to your project section:

:project:
  :test_threads: 8
  :compile_threads: 8

This should help speed things up. :)

[mvandervoord]

@swaldhoer -- Hm. I've done some digging into issue 2 in your list above (where the dependency headers weren't being found). Interesting enough, it appears that the generated gem file is correct.

I'm wondering if it's the upgrade process itself. Ceedling's self-upgrade feature is minimalist at this point (particularly compared to where we want it). Did the gem correctly update the contents of ../../tools/vendor/ceedling when you ran ceedling upgrade <projectname>? Or maybe you ran it yourself? Is ../../tools/vendor/ceedling/vendor/cmock populated with contents? Same with unity and cexception?

[swaldhoer]

I did not use the update mechanism. I created a new project and then copied everything. This is how we did it (1 or 2 times before I guess) and it worked. Doing so again, I saw changes in the vendored dependencies at tools/vendor/ceedling/vendor/cmock etc., so I assume this was fine.

[swaldhoer]

More concerning are the other failures. It's looking like something with the updated preprocessor calls isn't happy? Can you do a quick test where you disable the preprocessor and rake clobber test:all to see if it will complete a test then?

really rake or did you mean ceedling?

As a side note, feel free to add the following two lines to your project section:

:project:
  :test_threads: 8
  :compile_threads: 8

This should help speed things up. :)

I think I did it like this. I will test tomorrow again and come back.

[mvandervoord]

Hahaha... you are correct. I meant ceedling clobber test:all, not rake.

Your upgrade method should be fine. That suggests I have something wrong in the way ceedling is finding its subprojects when not using the gem. Thanks for the help! I should be able to find this error. :)

[Letme]

I am updating to newer version (because github default dockers now use Ruby 3.2) and just hit the following problem when using this branch.

I have following rakefile while Ceedling is submoduled to tools/ceedling:

PROJECT_CEEDLING_ROOT = "tools/ceedling"
load "#{PROJECT_CEEDLING_ROOT}/lib/ceedling/rakefile.rb"

task :default => %w[ test:all release ]

And the error I get after running command: rake -m -j 4 options:gcc clobber test:all (I have gcc.yml in targets/)

rake aborted!
NameError: uninitialized constant Configurator::Ceedling
*/tools/ceedling/lib/ceedling/configurator.rb:178:in `find_and_merge_plugins'
*/tools/ceedling/lib/ceedling/setupinator.rb:26:in `do_setup'
tools/ceedling/lib/ceedling/rakefile.rb:46:in `<top (required)>'
*/rakefile:5:in `load'
*/rakefile:5:in `<top (required)>'

Any ideas what is wrong?

[deltalejo]

Have you tried what is stated here? It seems that the rakefile should be something like

PROJECT_CEEDLING_ROOT = "tools/ceedling"
load "#{PROJECT_CEEDLING_ROOT}/lib/ceedling.rb"
Ceedling.load_project

task :default => %w[ test:all release ]

[mvandervoord]

@Letme are you calling rake instead of ceedling ?

[Letme]

@Letme are you calling rake instead of ceedling ?

Yes, I am. Because I do not install ceedling, i use it as submodule. I guess i need to call ./tools/ceedling/bin/ceedling then... (or something- sorry am on mobile)

[Letme]

OK, if I change rakefile with what @deltalejo suggested, it takes the local ceedling in submodule (but I also installed the above gem) I get it to build, but then building fails:

Building Objects
----------------
Compiling TestDSP_runner.c...
#<Thread:0x00007f4f4eb0f528 /*/tmp/mlx90632-library/tools/ceedling/lib/ceedling/par_map.rb:7 run> terminated with exception (report_on_exception is true):
/*/mlx90632-library/tools/ceedling/lib/ceedling/flaginator.rb:23:in `to_h': wrong element type String at 0 (expected array) (TypeError)

  hash.partition(&predicate).map(&:to_h)
                                 ^^^^^^
	from /*/mlx90632-library/tools/ceedling/lib/ceedling/flaginator.rb:23:in `map'
	from /*/mlx90632-library/tools/ceedling/lib/ceedling/flaginator.rb:23:in `partition'
	from /*/mlx90632-library/tools/ceedling/lib/ceedling/flaginator.rb:34:in `get_flag'
	from /*/mlx90632-library/tools/ceedling/lib/ceedling/flaginator.rb:71:in `flag_down'
	from /*/mlx90632-library/tools/ceedling/lib/ceedling/generator.rb:104:in `generate_object_file'
	from /*/mlx90632-library/tools/ceedling/lib/ceedling/test_invoker_helper.rb:61:in `block in generate_objects_now'
	from /*/mlx90632-library/tools/ceedling/lib/ceedling/par_map.rb:10:in `block (2 levels) in par_map'
rake aborted!
TypeError: wrong element type String at 0 (expected array)
/*/mlx90632-library/tools/ceedling/lib/ceedling/flaginator.rb:23:in `to_h'
/*mlx90632-library/tools/ceedling/lib/ceedling/flaginator.rb:23:in `map'
/*/mlx90632-library/tools/ceedling/lib/ceedling/flaginator.rb:23:in `partition'
/*/mlx90632-library/tools/ceedling/lib/ceedling/flaginator.rb:34:in `get_flag'
/*/mlx90632-library/tools/ceedling/lib/ceedling/flaginator.rb:71:in `flag_down'
/*/mlx90632-library/tools/ceedling/lib/ceedling/generator.rb:104:in `generate_object_file'
/*/mlx90632-library/tools/ceedling/lib/ceedling/test_invoker_helper.rb:61:in `block in generate_objects_now'
/*/mlx90632-library/tools/ceedling/lib/ceedling/par_map.rb:10:in `block (2 levels) in par_map'
Tasks: TOP => test:all

I also ran ceedling upgrade mlx90632-library, but also the ceedling in vendor repo fails.

File is: https://github.com/melexis/mlx90632-library/blob/master/test/TestDSP.c

[Letme]

OK, found the change. Flags definition was adjusted meanwhile so:

 :flags:
   :test:
     :compile:
      - -Wall

needs to become

 :flags:
   :test:
     :compile:
      :*:
        - -Wall

[Letme]

Ok, I think there is another dependency. First I had to install gem in docker to get even above modification within rakefile running (some strange include problems). Secondly the new feature from #740 now requires to have gdb installed as otherwise it outputs: ERROR: Config filepath [:tools][:backtrace_settings][:executable]['gdb'] does not exist in system search paths. even though I have put :use_backtrace_gdb_reporter: FALSE in my project.yml. Locally I have gdb installed and this all works.