cmake + icu

[bnoordhuis]

@srl295 I'm porting node's build system to cmake and I'd like you to take a look at making icutrim work. I've spent about two days trying to understand what all the scripts do but I'm not any closer.

Build with:

./configure --cmake=cmake && make -j8

The relevant build script is cmake/icu/common/CMakeLists.txt; cmake/icu/tools/CMakeLists.txt might be relevant too. Grep that first file for 'icu_small' to find the command.

I'm passing the same arguments as the gyp build but for flabbergasting reasons, the following happens:

1. iculslocs tries to open a bundle icudt60l-brkiter with ures_openDirect()
2. with the cmake build, it looks for that file (that doesn't exist) and fails
3. with the gyp build, it opens icudt60l.dat instead and continues - wut?

I established 2 and 3 by adding copious printf statements and strace -f. Yes, strace - I'm that desperate by now.

I tried to follow the logic inside ICU but whatever the cause (locale?), it's elusive and eluding. Hope you can help.

cc @mhdawson

[srl295]

OK first thing, for cmake, you might want to see http://bugs.icu-project.org/trac/ticket/7747

(this is on the source side, not data)

Specifically, gencmake.py generates IcuPaths.cmake from ICU's dependencies.txt (which has been included in ICU's source for a number of years). Then you can depend on a group such as stringenumeration and calculate the set of source files needed exactly, without the hacky inclusion/exclusion lists in icu-generic. Otherwise, ugh, you have yet another list of ICU source files that goes completely out of date if files move around.

[srl295]

It might be that iculslocs is fine, but your ICU build is broken. Opening icudt60l-brkiter and then falling back to icudt60l is exactly the right and normal behavior.

[bnoordhuis]

Opening icudt60l-brkiter and then falling back to icudt60l is exactly the right and normal behavior.

Okay, but what confounds me is that the gyp build doesn't even try to open icudt60l-brkitr, it goes straight for icudt60l (the one in out/Release, not the one in deps/icu-small.)

Conversely, the cmake iculslocs dies without even trying to open icudt60l. It looks for icudt60l-brkitr.dat/res_index.txt and icudt60l-brkitr.dat in that order and aborts. icudt60l.dat exists at that point, is a valid data file and is completely ignored.

Does that reproduce on your end?

OK first thing, for cmake, you might want to see http://bugs.icu-project.org/trac/ticket/7747

As long as it gives node enough control over the build I'm good with using the upstream solution. That hasn't been released though, or has it?

you have yet another list of ICU source files that goes completely out of date if files move around

True, but that doesn't worry me unduly, it's not much work.

[srl295]

Does that reproduce on your end?

I'm trying your branch out now.

That hasn't been released though, or has it?

icu/source/test/depstest/dependencies.txt has been released and gets tested and updates.

[bnoordhuis]

@srl295 Were you able to reproduce?

[srl295]

i got to this with cmake 3.9.4 and 3.10.2

$ ./configure --cmake=`which cmake`
…
CMake Error: Cannot determine link language for target "uv".
CMake Error: CMake can not determine linker language for target: uv

[bnoordhuis]

Interesting... what OS and cmake version is that?

[srl295]

it's mac high sierra , cmake version as listed.

[bnoordhuis]

Thanks, I'll take a look.

[srl295]

sorry! I was too terse.

$ ./configure --cmake=`which cmake`
creating icu_config.gypi
* Using ICU in deps/icu-small
creating icu_config.gypi
creating config.mk
creating config.gypi
{ 'target_defaults': { 'cflags': [],
                       'default_configuration': 'Release',
                       'defines': [],
                       'include_dirs': [],
                       'libraries': []},
  'variables': { 'asan': 0,
                 'coverage': 'false',
                 'debug_http2': 'false',
                 'debug_nghttp2': 'false',
                 'force_dynamic_crt': 0,
                 'host_arch': 'x64',
                 'icu_data_in': '/Users/srl/src/node/deps/icu-small/source/data/in/icudt60l.dat',
                 'icu_endianness': 'l',
                 'icu_gyp_path': 'tools/icu/icu-generic.gyp',
                 'icu_locales': 'en,root',
                 'icu_path': 'deps/icu-small',
                 'icu_small': 'true',
                 'icu_ver_major': '60',
                 'llvm_version': 0,
                 'node_byteorder': 'little',
                 'node_enable_d8': 'false',
                 'node_enable_v8_vtunejit': 'false',
                 'node_install_npm': 'true',
                 'node_module_version': 61,
                 'node_no_browser_globals': 'false',
                 'node_prefix': '/usr/local',
                 'node_release_urlbase': '',
                 'node_shared': 'false',
                 'node_shared_cares': 'false',
                 'node_shared_http_parser': 'false',
                 'node_shared_libuv': 'false',
                 'node_shared_nghttp2': 'false',
                 'node_shared_openssl': 'false',
                 'node_shared_zlib': 'false',
                 'node_tag': '',
                 'node_target_type': 'executable',
                 'node_use_bundled_v8': 'true',
                 'node_use_dtrace': 'true',
                 'node_use_etw': 'false',
                 'node_use_lttng': 'false',
                 'node_use_openssl': 'true',
                 'node_use_openssl_ca_store': 'false',
                 'node_use_perfctr': 'false',
                 'node_use_v8_platform': 'true',
                 'node_without_node_options': 'false',
                 'openssl_fips': '',
                 'openssl_no_asm': 0,
                 'shlib_suffix': '61.dylib',
                 'target_arch': 'x64',
                 'v8_enable_gdbjit': 0,
                 'v8_enable_i18n_support': 1,
                 'v8_enable_inspector': 1,
                 'v8_no_strict_aliasing': 1,
                 'v8_optimized_debug': 0,
                 'v8_promise_internal_field_count': 1,
                 'v8_random_seed': 0,
                 'v8_trace_maps': 0,
                 'v8_typed_array_max_size_in_heap': 0,
                 'v8_use_snapshot': 'true',
                 'want_separate_host_toolset': 0,
                 'xcode_version': '9.0'}}
-- Found PythonInterp: /Users/srl/src/node/out/tools/bin/python (found version "2.7.10") 
-- The C compiler identification is AppleClang 9.0.0.9000039
-- The CXX compiler identification is AppleClang 9.0.0.9000039
-- Check for working C compiler: /usr/local/opt/ccache/libexec/cc
-- Check for working C compiler: /usr/local/opt/ccache/libexec/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/local/opt/ccache/libexec/c++
-- Check for working CXX compiler: /usr/local/opt/ccache/libexec/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- The ASM compiler identification is AppleClang
-- Found assembler: /usr/local/opt/ccache/libexec/cc
-- Configuring done
CMake Error at cmake/uv/CMakeLists.txt:195 (add_library):
  Cannot find source file:

    /Users/srl/src/node/deps/uv/src/unix/pthread-barrier.c

  Tried extensions .c .C .c++ .cc .cpp .cxx .m .M .mm .h .hh .h++ .hm .hpp
  .hxx .in .txx


CMake Error: Cannot determine link language for target "uv".
CMake Error: CMake can not determine linker language for target: uv
-- Generating done
CMake Warning:
  Manually-specified variables were not used by the project:

    xcode_version


-- Build files have been written to: /Users/srl/src/node/out.cmake

[srl295]

grabbed the pthread-barrier.c from libuv upstream, able to continue edit and able to reproduce the issue