phobos
phobos copied to clipboard
cea-hpc
Installing from Source CentOS 7
Hey All,
I was recommended to give phobos a check as I set up to do a massive reorganization of our LTO library (both physically in our mechanical library and in our filing cabinet storage.
We're currently running CentOS 7 on the machine I have our LTO Tape Library attached to, so we can't use the RPM available (as mentioned in #2 ). I'd like to get it built from source, but the documentation is a bit vague in areas.
Right now, I have run ./autogen.sh which seemed to work nicely, followed by a ./configure which produces:
[...]
checking size of size_t... 8
checking for pkg-config... /usr/bin/pkg-config
checking pkg-config is at least version 0.9.0... yes
checking for GLIB2... no
configure: error: Package requirements (glib-2.0) were not met:
No package 'glib-2.0' found
Consider adjusting the PKG_CONFIG_PATH environment variable if you
installed software in a non-standard prefix.
Alternatively, you may set the environment variables GLIB2_CFLAGS
and GLIB2_LIBS to avoid the need to call pkg-config.
See the pkg-config man page for more details.
That said, the machine does have glib2 installed as well as pgsql-15 (as the distro provided pgsql is 9.2 which is under the version requirement), which is running.
I will note that the pgsql-15 library path doesn't have a pkconfig file in it, which may be a source of issue?
[root@eldin phobos]# ls /usr/pgsql-15/
bin/ lib/ share/
[root@eldin phobos]# ls /usr/pgsql-15/lib/
bitcode/ euc2004_sjis2004.so latin2_and_win1250.so libecpg.so.6 libpq.so.5.15 utf8_and_cyrillic.so utf8_and_euc_tw.so utf8_and_johab.so
cyrillic_and_mic.so euc_cn_and_mic.so latin_and_mic.so libecpg.so.6.15 libpqwalreceiver.so utf8_and_euc2004.so utf8_and_gb18030.so utf8_and_sjis2004.so
dict_int.so euc_jp_and_sjis.so libecpg_compat.so.3 libpgtypes.so.3 pgoutput.so utf8_and_euc_cn.so utf8_and_gbk.so utf8_and_sjis.so
dict_snowball.so euc_kr_and_mic.so libecpg_compat.so.3.15 libpgtypes.so.3.15 plpgsql.so utf8_and_euc_jp.so utf8_and_iso8859_1.so utf8_and_uhc.so
dict_xsyn.so euc_tw_and_big5.so libecpg.so libpq.so.5 utf8_and_big5.so utf8_and_euc_kr.so utf8_and_iso8859.so utf8_and_win.so
Any ideas?
~ Joe G.
Hi @JPGlaser
You will need to install development packages (-devel) to compile phobos. The configure script will give you a hint on what's missing. On Rocky Linux 9.2, I had to add at least the following devel packages to be able to compile phobos and build the rpms (with make rpm):
glib2-devel
jansson-devel
libattr-devel
libcmocka-devel
libini_config-devel
libpq-devel
protobuf-c-devel
sg3_utils-devel
Hope that helps!
Hello all,
Thanks @thiell for the answer, that actually seems to be the issue.
@JPGlaser, what is the state of the issue? Do you succeed in compiling Phobos? Is there any other missing package?
Hey @thiell and @SebaGougeaud ,
Sorry for the delay; last week was filled with holidays and I am just getting back to this. I was able to install everything above EXCEPT libpq. That is because for RHEL >7 repos, they changed the name of the package to postgresql15-libs and postgresql15-devel.
I can find the so library in /usr/pgsql-15/lib/libpq.so and the necessary includes in /usr/pgsql-15/include. However, pkconfig is not picking this up correctly resulting in:
checking for LIBPQ... no
configure: error: Package requirements (libpq >= 9.4) were not met:
No package 'libpq' found
Consider adjusting the PKG_CONFIG_PATH environment variable if you
installed software in a non-standard prefix.
Alternatively, you may set the environment variables LIBPQ_CFLAGS
and LIBPQ_LIBS to avoid the need to call pkg-config.
See the pkg-config man page for more details.
Running pg_config returns:
# pg_config
BINDIR = /usr/pgsql-15/bin
DOCDIR = /usr/pgsql-15/doc
HTMLDIR = /usr/pgsql-15/doc/html
INCLUDEDIR = /usr/pgsql-15/include
PKGINCLUDEDIR = /usr/pgsql-15/include
INCLUDEDIR-SERVER = /usr/pgsql-15/include/server
LIBDIR = /usr/pgsql-15/lib
PKGLIBDIR = /usr/pgsql-15/lib
LOCALEDIR = /usr/pgsql-15/share/locale
MANDIR = /usr/pgsql-15/share/man
SHAREDIR = /usr/pgsql-15/share
SYSCONFDIR = /etc/sysconfig/pgsql
PGXS = /usr/pgsql-15/lib/pgxs/src/makefiles/pgxs.mk
CONFIGURE = '--enable-rpath' '--prefix=/usr/pgsql-15' '--includedir=/usr/pgsql-15/include' '--mandir=/usr/pgsql-15/share/man' '--datadir=/usr/pgsql-15/share' '--libdir=/usr/pgsql-15/lib' '--with-lz4' '--with-zstd' '--enable-tap-tests' '--with-icu' '--with-llvm' '--with-perl' '--with-python' '--with-tcl' '--with-tclconfig=/usr/lib64' '--with-openssl' '--with-pam' '--with-gssapi' '--with-includes=/usr/include' '--with-libraries=/usr/lib64' '--enable-nls' '--enable-dtrace' '--with-uuid=e2fs' '--with-libxml' '--with-libxslt' '--with-ldap' '--with-selinux' '--with-systemd' '--with-system-tzdata=/usr/share/zoneinfo' '--sysconfdir=/etc/sysconfig/pgsql' '--docdir=/usr/pgsql-15/doc' '--htmldir=/usr/pgsql-15/doc/html' 'CFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic' 'LDFLAGS=-Wl,--as-needed' 'LLVM_CONFIG=/usr/lib64/llvm5.0/bin/llvm-config' 'CLANG=/opt/rh/llvm-toolset-7/root/usr/bin/clang' 'PKG_CONFIG_PATH=:/usr/lib64/pkgconfig:/usr/share/pkgconfig' 'PYTHON=/usr/bin/python3'
CC = gcc -std=gnu99
CPPFLAGS = -D_GNU_SOURCE -I/usr/include/libxml2 -I/usr/include Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic
CFLAGS_SL = -fPIC
LDFLAGS = -Wl,--as-needed -L/usr/lib64/llvm5.0/lib -L/usr/lib64 -Wl,--as-needed -Wl,-rpath,'/usr/pgsql-15/lib',--enable-new-dtags
Any ideas?
~ Joe G.
I was able to get phobos to finally compile after doing the following:
export PKG_CONFIG_PATH=/usr/pgsql-15/lib/pkgconfig:$PKG_CONFIG_PATH
In addition, I needed to do the following to gain access to the pg_* commands:
export PATH=/usr/pgsql-15/bin:$PATH
Finally, the compile required one additional library not previously mentioned in documentation:
yum install xxhash-devel
I have now run make and make install. Things seem to go well.
However, the command phobos does not seem to be present. Only phobosd, phobos_db, phobos_tape_library_mtx_test.bash, phobos_tape_library_test, phobos_tape_library_test.bash, and pho_ldm_helper are present in my $PATH. Additionally, testing phobos_db returns:
/usr/bin/python3: No module named phobos.db
Any idea folks?
~ Joe G.
Hi @JPGlaser
You will need to install development packages (-devel) to compile phobos. The configure script will give you a hint on what's missing. On Rocky Linux 9.2, I had to add at least the following devel packages to be able to compile phobos and build the rpms (with
make rpm):glib2-devel jansson-devel libattr-devel libcmocka-devel libini_config-devel libpq-devel protobuf-c-devel sg3_utils-develHope that helps!
As a direct follow-up to this, I tried to build an RPM via make rpm and things worked fine up till I got to the requirement that postgresql94-devel is needed, which is not supported on RHEL > 7. We only get:
Installed Packages
postgresql15-devel.x86_64 15.5-1PGDG.rhel7 @pgdg15
Available Packages
postgresql-devel.i686 9.2.24-8.el7_9 updates
postgresql-devel.x86_64 9.2.24-8.el7_9 updates
postgresql-pgpool-II-devel.x86_64 3.4.6-1.el7 epel
postgresql11-devel.x86_64 11.22-1PGDG.rhel7 pgdg11
postgresql12-devel.x86_64 12.17-1PGDG.rhel7 pgdg12
postgresql13-devel.x86_64 13.13-1PGDG.rhel7 pgdg13
postgresql14-devel.x86_64 14.10-1PGDG.rhel7 pgdg14
CentOS SCL does provide 9.5 as the lowest version (it use to have 9.4, but no longer does). However, this still breaks as it is provided as rh-postgresql95-postgresql-devel.
~ Joe G.
Regarding the missing phobos and phobos.db, I can reproduce it. This is probably due to some missing install rules in the autotool configuration files. The specfile runs specific commands to install all the necessary tools: make install and python setup.py install .... So this is expected behavior. You should favor the RPM over make install.
Regarding the RPM issues, postgresql-devel is only required for the build so it should not cause any issues while installing the RPM. If you can successfully create the RPM, you should be able to install it without postgresql-devel. It seems to me that postgresql-devel has been replaced by libpq-devel. So I guess you can just remove the BuildRequires: %{postgres_prefix}-devel from the specfile and retry make rpm. This is something that we will have to fix in the specfile. Let me know if this works.
I have found more information about the RPM issue. The package libpq-devel provides postgresql-devel which means that the build requires will actually install libpq-devel. If you run yum install postgresql-devel, yum will try to install libpq-devel (t least on CentOS 8.2 where I tried). So could you run yum list libpq-devel to see if you have the library available ?
On my machine, the package is in the repository appstream. So maybe you need to enable it. I have libpq-devel version 13.3 by the way. What version of CentOS are running ?
Regarding the missing
phobosandphobos.db, I can reproduce it. This is probably due to some missing install rules in the autotool configuration files. The specfile runs specific commands to install all the necessary tools:make installandpython setup.py install .... So this is expected behavior. You should favor the RPM overmake install.Regarding the RPM issues, postgresql-devel is only required for the build so it should not cause any issues while installing the RPM. If you can successfully create the RPM, you should be able to install it without postgresql-devel. It seems to me that postgresql-devel has been replaced by libpq-devel. So I guess you can just remove the
BuildRequires: %{postgres_prefix}-develfrom the specfile and retrymake rpm. This is something that we will have to fix in the specfile. Let me know if this works.
Gotcha; I'll focus on getting the RPM built then. I'll see what I can get done later today and update you all.
I have found more information about the RPM issue. The package
libpq-develprovidespostgresql-develwhich means that the build requires will actually installlibpq-devel. If you runyum install postgresql-devel, yum will try to installlibpq-devel(t least on CentOS 8.2 where I tried). So could you runyum list libpq-develto see if you have the library available ?On my machine, the package is in the repository appstream. So maybe you need to enable it. I have
libpq-develversion 13.3 by the way. What version of CentOS are running ?
So I am on CentOS7 with this server. We don't plan on migrating it to CentOS8 or over to Rocky 8 until it reaches EoL because of its core use as a ZFS and lustre headnode server. Here is a quick search for libpq:
# yum list libpq*
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: us.mirrors.virtono.com
* centos-sclo-rh: mirror.pit.teraswitch.com
* centos-sclo-sclo: ftp.osuosl.org
* epel: ftp.cse.buffalo.edu
* extras: us.mirrors.virtono.com
* rpmfusion-free-updates: mirror.math.princeton.edu
* rpmfusion-nonfree-updates: mirror.math.princeton.edu
* updates: mirrors.rit.edu
Available Packages
libpq5.x86_64 15.4-42PGDG.rhel7 pgdg-common
libpq5-devel.x86_64 15.4-42PGDG.rhel7 pgdg-common
libpqxx.x86_64 1:5.0.1-2.rhel7.1 pgdg11
libpqxx-devel.x86_64 1:5.0.1-2.rhel7.1 pgdg11
libpqxx-doc.noarch 1:4.0.1-1.el7 epel
So it looks like libpq 15 is available. What repo are you seeing libpq-devel available from?
~ Joe G.