How to build example?

[markwylde]

TL;DR; Solution down below (here).

Hey

This is probably a very beginner style question, as I'm in no way a C or CPP developer. But I'm looking at building the src/example file.

So I cloned a fresh ejdb instance, changed into that directory and ran (as specified in the README):

gcc -std=gnu11 -Wall -pedantic -c -o example1.o example1.c

Unfortunatly this doesn't build, I get:

$ gcc -std=gnu11 -Wall -pedantic -c -o example1.o example1.c
example1.c:2:10: fatal error: ejdb2.h: No such file or directory
    2 | #include <ejdb2.h>
      |          ^~~~~~~~~
compilation terminated.

So instead, I read online, you can include other directories when building. So I added an argument and ran:

gcc -I ../ -std=gnu11 -Wall -pedantic -c -o example1.o example1.c

This seems to get me further:

$ gcc -I ../ -std=gnu11 -Wall -pedantic -c -o example1.o example1.c
In file included from example1.c:2:
../ejdb2.h:31:10: fatal error: jql.h: No such file or directory
   31 | #include "jql.h"
      |          ^~~~~~~
compilation terminated.

So I add jql too, and it again get's me further. Although it does feel wrong to have to specify all these files manually as an -I, but I don't know anything about C.

But I then come to this error:

$ gcc -I ../jql -I ../ -std=gnu11 -Wall -pedantic -c -o example1.o example1.c
In file included from ../ejdb2.h:31,
                 from example1.c:2:
../jql/jql.h:40:10: fatal error: iowow/iwjson.h: No such file or directory
   40 | #include <iowow/iwjson.h>
      |          ^~~~~~~~~~~~~~~~
compilation terminated.

Now I can't really add -I ../../extra/iowow because I think the actual path is extra/iowow/src/json.

Anyway, I tried it and it failed as expected:

$ gcc -I ../../extra/iowow -I ../jql -I ../ -std=gnu11 -Wall -pedantic -c -o example1.o example1.c
In file included from ../ejdb2.h:31,
                 from example1.c:2:
../jql/jql.h:40:10: fatal error: iowow/iwjson.h: No such file or directory
   40 | #include <iowow/iwjson.h>
      |          ^~~~~~~~~~~~~~~~
compilation terminated.

I'm guessing this is why you/we/c developers use cmake because it finds and includes all these directories for us?

Anyway, is it too troubling for me to ask you to point me in the right direction?

Should it be possible to build the example just using gcc alone, with a few includes?

Is there another argument/arguments I should be providing?

If it would take too long to explain all this, can you send me some guides/getting started material. I've searched Google and I guess I'm struggling with what even to search for.

[markwylde]

Yeah, the more I look into this, the more certain I am missing something fundamental to how C developers do things.

I'm looking at the src/jql/jql.h file and can see the include below:

#include <iowow/iwjson.h>
#include <iowow/iwlog.h>

But the actual files should be: iowow/src/json/iwjson.h and iowow/src/log/iwlog.h.

So then I look at the CMakeLists.txt in iowow, and see there is this line:

list(APPEND PUB_HDRS ${CMAKE_CURRENT_SOURCE_DIR}/basedefs.h
                     ${CMAKE_CURRENT_SOURCE_DIR}/fs/iwdlsnr.h
                     ...
                     ${CMAKE_CURRENT_SOURCE_DIR}/log/iwlog.h

So am I right in saying this basically merged all these files into one virtual directory called iowow?

I'm guessing that all CMake really does is compile a huge command line prompt, with a bunch of include mappings? But I don't understanding how the c compiler maps iowow/iwjson.h to iowow/src/json/iwjson.h.

So I thought maybe CMake does some symlinking magic in that build directory. So I was expecting maybe a huge monolithic ./build/src/iowow/... folder with every flattened?

[markwylde]

Okay, I may have gone down a bit of a rabbit hole here, but think I might be on to something.

Would I be right in saying, the command below was for last years version of ejdb:

gcc -std=gnu11 -Wall -pedantic -c -o example1.o example1.c

And now, since iowow and iwnet have been abstracted into their own directories, we now need to link to the built external libraries when compiling?

So, I went and compiled iowow separately and got the iowow.so file.

But still no joy:

$ gcc -l../../extra/iowow/build/src/libiowow-1.a -I ../jql -I ../ -std=gnu11 -Wall -pedantic -c -o example1.o example1.c
In file included from ../ejdb2.h:31,
                 from example1.c:2:
../jql/jql.h:40:10: fatal error: iowow/iwjson.h: No such file or directory
   40 | #include <iowow/iwjson.h>
      |          ^~~~~~~~~~~~~~~~
compilation terminated.

[markwylde]

So finally I caved and installed the iowow-1.4.16-Release-Linux-x86_64.deb package on my OS.

I feel like I should have been able to somehow tell gcc where to find that .so or .a file, but it's getting late and I'm close to giving up :sob:

But installing in into my OS seemed to get me another step closer:

$ gcc -I ../jql -I ../ -std=gnu11 -Wall -pedantic -o example1.o example1.c
/usr/bin/ld: /tmp/cclHt5nT.o: in function `documents_visitor':
example1.c:(.text+0x2f): undefined reference to `jbl_fstream_json_printer'
/usr/bin/ld: example1.c:(.text+0x37): undefined reference to `jbl_as_json'
/usr/bin/ld: /tmp/cclHt5nT.o: in function `main':
example1.c:(.text+0x9e): undefined reference to `ejdb_init'
/usr/bin/ld: example1.c:(.text+0xd6): undefined reference to `iwlog3'
/usr/bin/ld: example1.c:(.text+0xf9): undefined reference to `ejdb_open'
/usr/bin/ld: example1.c:(.text+0x131): undefined reference to `iwlog3'
/usr/bin/ld: example1.c:(.text+0x154): undefined reference to `jbl_from_json'
/usr/bin/ld: example1.c:(.text+0x196): undefined reference to `ejdb_put_new'
/usr/bin/ld: example1.c:(.text+0x1c3): undefined reference to `jbl_destroy'
/usr/bin/ld: example1.c:(.text+0x1dc): undefined reference to `jbl_from_json'
/usr/bin/ld: example1.c:(.text+0x21e): undefined reference to `ejdb_put_new'
/usr/bin/ld: example1.c:(.text+0x24b): undefined reference to `jbl_destroy'
/usr/bin/ld: example1.c:(.text+0x26b): undefined reference to `jql_create'
/usr/bin/ld: example1.c:(.text+0x336): undefined reference to `jql_set_i64'
/usr/bin/ld: example1.c:(.text+0x35f): undefined reference to `ejdb_exec'
/usr/bin/ld: example1.c:(.text+0x393): undefined reference to `jql_destroy'
/usr/bin/ld: example1.c:(.text+0x3ae): undefined reference to `jbl_destroy'
/usr/bin/ld: example1.c:(.text+0x3bd): undefined reference to `ejdb_close'
/usr/bin/ld: example1.c:(.text+0x3ee): undefined reference to `iwlog3'
collect2: error: ld returned 1 exit status

But now I'm just lost and think I'm so far down the rabbit hole, and I'm too out of my depth.

[markwylde]

Ahh, Google has helped. Apparently you need to specify the -l library link after the file is included.

This got me to the following:

gcc -I ../jbl -I ../jql -I ../ -std=gnu11 -Wall -pedantic -o example1.o example1.c -liowow
/usr/bin/ld: /tmp/ccNU7wuE.o: in function `main':
example1.c:(.text+0x9e): undefined reference to `ejdb_init'
/usr/bin/ld: example1.c:(.text+0xf9): undefined reference to `ejdb_open'
/usr/bin/ld: example1.c:(.text+0x196): undefined reference to `ejdb_put_new'
/usr/bin/ld: example1.c:(.text+0x21e): undefined reference to `ejdb_put_new'
/usr/bin/ld: example1.c:(.text+0x26b): undefined reference to `jql_create'
/usr/bin/ld: example1.c:(.text+0x336): undefined reference to `jql_set_i64'
/usr/bin/ld: example1.c:(.text+0x35f): undefined reference to `ejdb_exec'
/usr/bin/ld: example1.c:(.text+0x393): undefined reference to `jql_destroy'
/usr/bin/ld: example1.c:(.text+0x3bd): undefined reference to `ejdb_close'
collect2: error: ld returned 1 exit status

Now, it turns out ejdb itself can be built (as per the README) as a debian package. So, I built that and installed it:

dpkg -i ./build/ejdb2-2.73-Release-Linux-x86_64.deb

Then ran:

gcc -I ../jbl -I ../jql -I ../ -std=gnu11 -Wall -pedantic -o example1.o example1.c -liowow -lejdb2

Boom! No errors.

That is, of course, until I try and run the outputted binary:

$ ./example1.o 
./example1.o: error while loading shared libraries: libejdb2.so.2: cannot open shared object file: No such file or directory

Anyway, I think I'm done and it's getting late. I appreciate any time you could give to point me in the right direction, but of course understand if this is far far from the scope of this project. Thanks anyway! :)

[markwylde]

Oh, apparently this worked:

LD_LIBRARY_PATH="../../../ejdb2/build/src/" ./example1.o 

But why do I have to specify LD_LIBRARY_PATH. I don't even want this as a shared library. I just want to statically link the whole thing.

Okay turning laptop off :P

[markwylde]

Just for future reference, if anyone else is trying to do this, I've got a little further this morning with the static linking. Dynamic linking seems to be working totally fine.

So far, my progress with statically linking is below:

I should also note, at this stage, I have no idea if the debian packages actually need to be installed. I have a feeling specifying the .a files below, might mean we don't need to install the dpkg.

Steps:

1. mkdir /projects && cd /projects
2. git clone https://github.com/Softmotions/iowow.git
3. git clone https://github.com/Softmotions/iwnet.git

Then follow the README.md in each of the projects to build the project using the official instructions.

This will create a bunch of .so files that can be statically linked. (I think).

The following (after a lot of trial and error) has got me pretty close. There's just one library I have no idea what it is yet that I don't think it can find.

gcc \
  -I ../jbl \
  -I ../jql \
  -I ../ \
  -std=gnu11 \
  -Wall \
  -pedantic \
  -o example1.o \
  example1.c \
  -L /projects/ejdb/build/_CPack_Packages/Linux/DEB/ejdb2-2.73-Release-Linux-x86_64/usr/local/lib/ \
  -l:libejdb2-2.a \
  -L /projects/iowow/build/_CPack_Packages/Linux/DEB/iowow-1.4.16-Release-Linux-x86_64/usr/lib \
  -l:libiowow-1.a \
  -L /projects/iwnet/build/lib \
  -l:libiwnet-1.a \
  -lm

Current output is below, but this has gone down from around 200 lines without the ejdb, iowow and iwnet libraries being specified.

/usr/bin/ld: /projects/ejdb/build/_CPack_Packages/Linux/DEB/ejdb2-2.73-Release-Linux-x86_64/usr/local/lib//libiwnet-1.a(iwn_poller.c.o): in function `iwn_poller_destroy':
iwn_poller.c:(.text+0x132b): undefined reference to `iwtp_shutdown'
/usr/bin/ld: /projects/ejdb/build/_CPack_Packages/Linux/DEB/ejdb2-2.73-Release-Linux-x86_64/usr/local/lib//libiwnet-1.a(iwn_poller.c.o): in function `iwn_poller_create':
iwn_poller.c:(.text+0x1465): undefined reference to `iwtp_start'
/usr/bin/ld: iwn_poller.c:(.text+0x15a5): undefined reference to `iwtp_shutdown'
/usr/bin/ld: /projects/ejdb/build/_CPack_Packages/Linux/DEB/ejdb2-2.73-Release-Linux-x86_64/usr/local/lib//libiwnet-1.a(iwn_poller.c.o): in function `iwn_poller_poll':
iwn_poller.c:(.text+0x1a9e): undefined reference to `iwtp_schedule'
/usr/bin/ld: /projects/ejdb/build/_CPack_Packages/Linux/DEB/ejdb2-2.73-Release-Linux-x86_64/usr/local/lib//libiwnet-1.a(iwn_poller.c.o): in function `iwn_poller_task':
iwn_poller.c:(.text+0x1689): undefined reference to `iwtp_schedule'
collect2: error: ld returned 1 exit status

[markwylde]

YES!!!!!!! Success! I've managed to build example statically. The last problem was simply an ordering issue of the linked libraries.

The following command works, and successfully builds the example file.

I have tested this in an ubuntu docker container, which doesn't have any of the dpkg installed. Works perfectly!

Preparation

You will need to build (or alternatively source) three libraries (at the current time of posting):

• ejdb (this repo)
• libiwnet (https://github.com/Softmotions/iwnet)
• iowow (https://github.com/Softmotions/iowow)

This was very easy and just worked out of the box, following the instructions in the README.md files, inside the respective repos.

Once this was done I just searched for all the .so files in the build directory.

Build

gcc \
  -I ../jbl \
  -I ../jql \
  -I ../ \
  -std=gnu11 \
  -Wall \
  -pedantic \
  -o example1.o \
  example1.c \
  -L /projects/ejdb/build/_CPack_Packages/Linux/DEB/ejdb2-2.73-Release-Linux-x86_64/usr/local/lib/ \
  -l:libejdb2-2.a \
  -L /projects/iwnet/build/lib \
  -l:libiwnet-1.a \
  -L /projects/ejdb/extra/iowow/build/_CPack_Packages/Linux/DEB/iowow-1.4.16-Release-Linux-x86_64/usr/lib \
  -l:libiowow-1.a \
  -lm

Run

docker run -itv `pwd`:/app ubuntu bash
./example1.o

Output

$ ./example1.o 
{
 "name": "Darko",
 "age": 8
}{
 "name": "Bianca",
 "age": 4
}

[adamansky]

Hi Mark! Great road! Definitive will refer this thread as how-to!) Anyway docs should be improved, but still have no time