cfarm icon indicating copy to clipboard operation
cfarm copied to clipboard

Introduce the concept of a matrix worker.

Open robertmaynard opened this issue 10 years ago • 0 comments

A standard deployment file ( .cdep ) is written to target a single compiler, configuration and library type. This is great for testing projects with very specific setups, but isn't great when you want to verify a project against all combinations.

This is why we should introduce the concept of the matrix to deployment files. Instead of having a worker target a single element of the compiler, configuration, and library matrix, why not have a worker that is meant to verify all elements.

For the initial pass we should limit the matrix to 3 dimensions, which are compiler, configuration type, and library type. I envision that we could easily extend the .cdep file to support a matrix by extending the syntax for compiler, configuration, and library type as follows:

{
"hostname" : "bigboard",
"user" : "hiro",
"compilers" : [  
   {"name" : "legacy_gcc",  "c" : "/usr/bin/gcc-4.6", "cpp" : "/usr/bin/g++-4.6"},
   {"name" : "gcc4.8",  "c" : "/usr/bin/gcc-4.8", "cpp" : "/usr/bin/g++-4.8"},
   {"name" : "clang",   "c" : "/usr/bin/clang",   "cpp" : "/usr/bin/clang++"}
   ]
"src_location" : "/home/hiro/Work/bigboard/src",
"build_location" : "/home/hiro/Work/bigboard/build",
"build_generator" : "Ninja",
"build_flags" : "-j8",
"build_configuration" : [ "debug", "release" ]
"library_type" : [ "static", "shared" ]
}

Now what this means is that cfarm will create multiple build directories under the `build_location``, each labelled to identify the compiler, configuration and library type it represents. For the above example we would generate the following folders

/home/hiro/Work/bigboard/build/
 - legacy_gcc_debug_static/
 - legacy_gcc_debug_shared/
 - legacy_gcc_release_static/
 - legacy_gcc_release_shared/
 - gcc4.8_debug_static/
 - gcc4.8_debug_shared/
 - gcc4.8_release_static/
 - gcc4.8_release_shared/
 - clang_debug_static/
 - clang_debug_shared/
 - clang_release_static/
 - clang_release_shared/

Because of the explosion of build directories we need to add 3 quality of life improvements.

  1. Give each worker a pretty_name this will be a combination of the deployment file name plus the build director for the worker so for example a pretty_name could be linux_clang_debug_static.
  2. Override fabric.utils.puts to use a workers pretty_name instead of the current hostname
  3. Add support to cfarm commands to invoke workers based on their pretty_name and on their deployment name. This will allow a user to test a single component of the matrix.

robertmaynard avatar May 20 '14 17:05 robertmaynard