build-sys,cmake: Add support for building with CMake

[ArcsinX]

This patch adds support for building with CMake.

I am not sure that it covers all cases, but I've checked it with:

• GCC (Ubuntu 18.04)
• MinGW (Windows 10)
• Visual Studio (Windows 10)

[codecov[bot]]

Codecov Report

Merging #2959 (f87e292) into master (4a3ba77) will increase coverage by 0.10%. The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #2959      +/-   ##
==========================================
+ Coverage   87.25%   87.35%   +0.10%     
==========================================
  Files         194      199       +5     
  Lines       44435    47689    +3254     
==========================================
+ Hits        38770    41661    +2891     
- Misses       5665     6028     +363     

Impacted Files	Coverage Δ
parsers/yaml.c	92.06% <0.00%> (-6.25%)	:arrow_down:
main/field.c	92.61% <0.00%> (-4.86%)	:arrow_down:
main/lregex.c	83.60% <0.00%> (-4.40%)	:arrow_down:
parsers/cxx/cxx_parser_function.c	92.46% <0.00%> (-0.17%)	:arrow_down:
parsers/cxx/cxx_parser_tokenizer.c	92.45% <0.00%> (-0.09%)	:arrow_down:
main/flags.c	100.00% <0.00%> (ø)
optlib/pod.c	100.00% <0.00%> (ø)
main/seccomp.c	0.00% <0.00%> (ø)
dsl/qualifier.c	73.17% <0.00%> (ø)
optlib/qemuhx.c	100.00% <0.00%> (ø)
... and 25 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 4a3ba77...f87e292. Read the comment docs.

[leleliu008]

thanks for your contribution, but a lot of features are missing.

[leleliu008]

I have been working on supporting CMake recently, I will make a new PR tomorrow.

[leleliu008]

our build system should support GNU/Linux macOS Windows FreeBSD NetBSD OpenBSD Cygwin msys2, cross-compiling for Android and mingw, all these platform should be tested.

[masatake]

Why we should use CMake?

[leleliu008]

@masatake

1. cross-platform
2. CMake is more simpler than autotools, but I don't like CMake syntax, I think it can be more simper
3. cmake can gen ninja config files but autotools can only gen makefiles

[leleliu008]

Hi, @masatake , don't you need sleep? It seems that you are working 24 hours every day. 🤠

[masatake]

Hi, @masatake , don't you need sleep? It seems that you are working 24 hours every day.

I really want to finish the "optscript" branch. Till merging it, I will work hard as much as possible!

@leleliu008, @k-takata, have you ever used CMake ? This project is highly depends on you, experienced developers who know build-system and scripts well. If all you are positive about switching to CMake, I don't object to the switch.

In addition, I wonder how Geany developers say about using CMake.

#2741 and #2794 touch the build scripts. If we switch to CMake, the pull request must be revised. As I wrote I'm working hard for merging #2794. However, it will take more time.

[ArcsinX]

I think that cmake can be added in addition to existing build systems as experimental feature with future improvements (i.e. we should not replace existing build systems for now). And after some time when everyone will be sure that it works for all needed architectures/OS with different configuration options, switch is possible.

[leleliu008]

I think that cmake can be added in addition to existing build systems as experimental feature with future improvements (i.e. we should not replace existing build systems for now). And after some time when everyone will be sure that it works for all needed architectures/OS with different configuration options, switch is possible.

Yes.

1. The wheel of history rolls on, the build system should move forward along with the time. CMake is a mordern C/C++ build system. It is widely used by new C/C++ developers. I think we should introduce a mordern C/C++ build system as an alternative. we don't want to replace the old one but want to coexist with the old one until we decide to switch to it.

2. I have been working on supporting CMake, GNU/Linux macOS Windows FreeBSD NetBSD OpenBSD Cygwin msys2, cross-compiling for Android and mingw, all these platform have been tested. now, I'm optimizing the CMake script.

3. I will maintain it in my fork nowdays.

[leleliu008]

I known some people hope our project support CMake build system. Why these people want this feature?

I think they don't want to install automake autoconf, newer programmers usually don't known autotools build system well, because they beginning with CMake, CMake is supported by most popular programing platforms(Android NDK, MSVC, etc). User don't need to install CMake on these platforms, because CMake has been included by these platforms. To resolve this problem, I have wrote a powerful autogen.sh, it can help user to install all softwares used by this project. no any warnings. let me show you how to use it:

git clone https://github.com/universal-ctags/ctags
git clone https://github.com/leleliu008/autogen.sh

cp autogen.sh/autogen.sh ctags
cp autogen.sh/autogen.rc ctags

cd ctags
./autogen.sh
./configure [options]
make
[sudo ] make install

Our project can build with CMake, it is supported by me, following code show you how to use it:

git clone https://github.com/universal-ctags/ctags
git clone https://github.com/leleliu008/cmakew

cp -r cmakew/* ctags/

cd ctags
patch -p1 < $(ls patch/*)
./cmakew config [options]
./cmakew build
[sudo] ./cmakew install

cmakew likes autogen.sh, it can help user to install all softwares used by this project.

About more details of CMake Build System, please read me

[masatake]

If we can make our build scripts simpler, I would like to switch to CMake.

One of what I don't like in the current Autotools-based build scripts is the way to add a man page is very complicated. Can CMake improve this? I would like to avoid using both Autotools and CMake. Using both may make our build scripts more complicated.

[leleliu008]

1. I'm not very clear the way to add a man page in the current Autotools-based build scripts. Could you show me some details? In CMake, the workfolow is *.rst.in -> *.rst -> manpages

2. simpler or not is such a subjective matter that different people has different taste. we should practice and never hearing someone saying it is good or not, because that's not our taste.

3. We should aware that there is a certain amount of risk if we switch to CMake. The best practice is let CMake coexist with the old one for some time till we decide to switch to it.

[leleliu008]

You known I wrote a package manager called ndk-pkg, 270 common used packages are ported. About 160 packages using Autotools/configure build system, they usually live over ten years and developed by GNU. About 60 packages using CMake, they are vey young projects. About 10 packages using Meson, they are also very young projects. Some Autotools/configure based project start to support CMake or Meson.

[leleliu008]

As we all known, in software engineering area, there are no silver bullets, every build system has it's problem. For example, many people(include me) don't like the syntax of CMake. I plan to develop a new programming language as simple as POSIX Shell to replace the syntax of CMake.

[masatake]

1. I'm not very clear the way to add a man page in the current Autotools-based build scripts. Could you show me some details? In CMake, the workfolow is *.rst.in -> *.rst -> manpages

When adding a man page I have to update following places:

• man_MANS in Makefile.am
• list inside AC_CONFIG_FILES
• IN_SOURCE_FILES or SOURCE_FILES in man/Makefile
• docs/man-pages.rst

See https://github.com/universal-ctags/ctags/commit/31e07a30b48373cb3a03c7002d6baf285b54383d#diff-b19f04b8a1afe9c89d21aa97fc6cffdb32cb71940d13aa0084acf30381c0ac07 .

If I can reduce the places for adding, I'm quite happy.

2. simpler or not is such a subjective matter that different people has different taste. we should practice and never hearing someone saying it is good or not, because that's not our taste.

I would like to reduce the duplicated list of files. This is one of my criteria of simplicity. I want the steps for introducing a code generator like optlib2c and packcc are fewer. As far as I know CMake has ability to eliminate ctags_vs2013.vcxproj. Am I correct? If the answer is yes, I would like to switch to CMake.

3. We should aware that there is a certain amount of risk if we switch to CMake. The best practice is let CMake coexist with the old one for some time till we decide to switch to it.

Do you have a plan?

I will try to write a "map" that can be used as the start of discsssion.

Our build scripts have following aspects:

• making binaries: ctags, readtags, optscript, ... caring with
  • unix flavour platform
  • window platform
  • cross compiling
  • code translators
• making document
  • man pages
  • docs.ctags.io
• running test
  • units, tmain,... (running on CI/CD evnronment, or local pc)
  • roundtrip...(running on CI/CD evnronment, or local pc)
  • codecheck...(running on CI/CD evnronment, or local pc)
  • distcheck (running on CI/CD evnronment, or local pc)
  • noise, chop, slap (running on local pc)
  • validate-input (running on local pc, but I don't use this much)
• ctags can be built as part of Geany.

[leleliu008]

@masatake

1. I think many lines in source.mak Makefile.am can be decreased via adjusting directory structure and use wildcard notdir patsubst Makefile functions, I don't known if bmake support these functions. For example:

man_MANS = \
    man/ctags.1 \
    man/readtags.1 \
    \
    man/tags.5 \
    \
    man/ctags-client-tools.7 \
    man/ctags-incompatibilities.7 \
    man/ctags-optlib.7 \
    man/ctags-faq.7 \
    \
    man/ctags-lang-julia.7 \
    man/ctags-lang-python.7 \
    man/ctags-lang-verilog.7 \
    man/ctags-lang-inko.7 \
    man/ctags-lang-r.7 \
    \
    $(NULL)

above code can be changed to following:

man_MANS = $(patsubst %.rst.in, %, $(wildcard man/*.rst.in))

CMake has similar commands.

If someone want to add a new manpage, just create a new file named xx.N.rst.in and place it in man directory, the rest things give build system to handle, so easy.

2. Yes, CMake can generates almost all platform config files, support list

3. If you are interesting in introducing CMake build system to our project, I hope you can give it a try and then give me some suggests. the guide

4. building and installing on GNU/Linux macOS Windows FreeBSD NetBSD OpenBSD Cygwin msys2, cross-compiling for Android and mingw, all these platform have been tested using GitHub Actions.

5. pack source package and binary package are supported.

6. test case only support test man and test roundtrip, other tests will be supported soon.

7. generating docs.ctags.io have not been supported yet, but I think It is very easy, I will support it soon. I think it can be split as a separated project.

8. I find geany using autotools, if ctags as a part of Geany, our project should keep supporting autotools.

[ArcsinX]

Our project can build with CMake, it is supported by me, following code show you how to use it:

CMake was desined to avoid using compilcated parametrised shell scripts as a part of the project, i.e. shell scripts on the top of CMake as a part of the project is a very bad solution.

For example, many people(include me) don't like the syntax of CMake. I plan to develop a new programming language as simple as POSIX Shell to replace the syntax of CMake.

It does not look as a good enough reason to reinvent the wheel.

@masatake

As far as I know CMake has ability to eliminate ctags_vs2013.vcxproj. Am I correct?

So, we can do this and I suggest to remove ctags_vs2013.* (and maybe mk_mvc.mak? ) and add initial basic CMake support (like in this MR). Thus, we are not adding a new one build system item, but replacing one with another (Visual Studio project => CMake). After that we can improve CMake build system (tests, features, documentation, etc) in separate MRs and at some point, when everybody will be happy with functionallity, we can switch.

[masatake]

@ArcsinX, could you put your copyright notice to your CMakeList.txt like configure.ac?

Quoted from configure.ac:

#	Copyright (c) 2009, Darren Hiebert
#
#	This source code is released for free distribution under the terms
#	of the GNU General Public License version 2 or (at your option) any
#	later version.

[masatake]

Thanks but... you are not "Darren Hiebert"! This year is 2021.

[masatake]

Thank you.

[eranif]

sorry to resurrect this PR, is there a chance that this PR will be merged any time soon? this will add the possibility of building u-ctags as an external git submodule by other projects that require u-ctags the current build system autoconf/configure is out dated and does not play nice with other open source projects out there

[masatake]

sorry to resurrect this PR, is there a chance that this PR will be merged any time soon?

I have no plan to merge this.

[eranif]

any particular reason? it does not break the code, and it actually adds a modern time build system for those who choose to use it

[masatake]

any particular reason? it does not break the code, and it actually adds a modern time build system for those who choose to use it

I don't want to maintain two sets of build scripts. The scripts for Autotools already work well.