bigtop icon indicating copy to clipboard operation
bigtop copied to clipboard

Published 20 hours ago • verified publisher icon apache

BIGTOP-3504. Split the Phoenix package into client and server.

Open sekikn opened this issue 4 years ago • 4 comments

https://issues.apache.org/jira/browse/BIGTOP-3504

sekikn avatar Feb 23 '21 23:02 sekikn

I manually tested this PR on CentOS 7:

$ ./gradlew phoenix-clean phoenix-pkg repo

...

BUILD SUCCESSFUL in 11m 18s
9 actionable tasks: 9 executed
$ rpm -qlp output/phoenix/noarch/phoenix-5.1.0-1.el7.noarch.rpm 
/etc/default/phoenix
/usr/lib/phoenix/phoenix-pherf-5.1.0.jar
/usr/lib/phoenix/phoenix-server-hbase-2.2-5.1.0.jar
/usr/lib/phoenix/phoenix-server.jar
$ rpm -qlp output/phoenix/noarch/phoenix-client-5.1.0-1.el7.noarch.rpm 
/etc/phoenix/conf.dist
/usr/lib/phoenix/bin
/usr/lib/phoenix/bin/argparse-1.4.0
/usr/lib/phoenix/bin/argparse-1.4.0/argparse.py
/usr/lib/phoenix/bin/daemon.py
/usr/lib/phoenix/bin/end2endTest.py
/usr/lib/phoenix/bin/hadoop-metrics2-hbase.properties
/usr/lib/phoenix/bin/hadoop-metrics2-phoenix.properties
/usr/lib/phoenix/bin/hbase-omid-client-config.yml
/usr/lib/phoenix/bin/hbase-site.xml
/usr/lib/phoenix/bin/log4j.properties
/usr/lib/phoenix/bin/performance.py
/usr/lib/phoenix/bin/pherf-standalone.py
/usr/lib/phoenix/bin/phoenix_utils.py
/usr/lib/phoenix/bin/psql.py
/usr/lib/phoenix/bin/readme.txt
/usr/lib/phoenix/bin/sqlline.py
/usr/lib/phoenix/bin/traceserver.py
/usr/lib/phoenix/phoenix-client-hbase-2.2-5.1.0.jar
/usr/lib/phoenix/phoenix-client.jar
/usr/share/doc/phoenix-5.1.0
/usr/share/doc/phoenix-5.1.0/LICENSE
/usr/share/doc/phoenix-5.1.0/NOTICE
/usr/share/doc/phoenix-5.1.0/examples
/usr/share/doc/phoenix-5.1.0/examples/STOCK_SYMBOL.csv
/usr/share/doc/phoenix-5.1.0/examples/STOCK_SYMBOL.sql
/usr/share/doc/phoenix-5.1.0/examples/WEB_STAT.csv
/usr/share/doc/phoenix-5.1.0/examples/WEB_STAT.sql
/usr/share/doc/phoenix-5.1.0/examples/WEB_STAT_QUERIES.sql
$ cd provisioner/docker
$ ./docker-hadoop.sh -d -k hdfs,hbase -L -C config_centos-7.yaml -c 3

...

$ for i in $(seq 1 3); do
>   if [ $i -eq 1 ]; then ./docker-hadoop.sh --exec $i yum install -y phoenix-client; fi
>   ./docker-hadoop.sh --exec $i yum install -y phoenix
>   ./docker-hadoop.sh --exec $i ln -s /usr/lib/phoenix/phoenix-server.jar /usr/lib/hbase/lib/
>   ./docker-hadoop.sh --exec $i systemctl restart hbase-regionserver
>   if [ $i -eq 1 ]; then ./docker-hadoop.sh --exec $i systemctl restart hbase-master; fi
> done

...

$ ./docker-hadoop.sh --exec 1 /usr/lib/phoenix/bin/sqlline.py

...

Connected to: Phoenix (version 5.1)
Driver: PhoenixEmbeddedDriver (version 5.1)
Autocommit status: true
Transaction isolation: TRANSACTION_READ_COMMITTED
sqlline version 1.9.0
0: jdbc:phoenix:> CREATE TABLE t(c VARCHAR PRIMARY KEY);
No rows affected (1.566 seconds)
0: jdbc:phoenix:> UPSERT INTO t VALUES ('foo');
1 row affected (0.164 seconds)
0: jdbc:phoenix:> SELECT * FROM t;
+-----+
|  C  |
+-----+
| foo |
+-----+
1 row selected (0.077 seconds)

and Debian 10:

$ ./gradlew phoenix-clean phoenix-pkg repo

...

BUILD SUCCESSFUL in 29m 55s
9 actionable tasks: 9 executed
$ dpkg -c output/phoenix/phoenix_5.1.0-1_all.deb 
drwxr-xr-x root/root         0 2021-02-23 15:03 ./
drwxr-xr-x root/root         0 2021-02-23 15:03 ./etc/
drwxr-xr-x root/root         0 2021-02-23 15:03 ./etc/default/
-rw-r--r-- root/root       993 2021-02-23 15:03 ./etc/default/phoenix
drwxr-xr-x root/root         0 2021-02-23 15:03 ./usr/
drwxr-xr-x root/root         0 2021-02-23 15:03 ./usr/lib/
drwxr-xr-x root/root         0 2021-02-23 15:03 ./usr/lib/phoenix/
-rw-r--r-- root/root   3679575 2021-02-23 15:03 ./usr/lib/phoenix/phoenix-pherf-5.1.0.jar
-rw-r--r-- root/root  67085723 2021-02-23 15:03 ./usr/lib/phoenix/phoenix-server-hbase-2.2-5.1.0.jar
drwxr-xr-x root/root         0 2021-02-23 15:03 ./usr/share/
drwxr-xr-x root/root         0 2021-02-23 15:03 ./usr/share/doc/
drwxr-xr-x root/root         0 2021-02-23 15:03 ./usr/share/doc/phoenix/
-rw-r--r-- root/root       131 2021-02-23 15:03 ./usr/share/doc/phoenix/changelog.Debian.gz
-rw-r--r-- root/root     24257 2021-02-05 08:27 ./usr/share/doc/phoenix/changelog.gz
-rw-r--r-- root/root       318 2021-02-23 15:03 ./usr/share/doc/phoenix/copyright
lrwxrwxrwx root/root         0 2021-02-23 15:03 ./usr/lib/phoenix/phoenix-server.jar -> phoenix-server-hbase-2.2-5.1.0.jar
$ dpkg -c output/phoenix/phoenix-client_5.1.0-1_all.deb 
drwxr-xr-x root/root         0 2021-02-23 15:03 ./
drwxr-xr-x root/root         0 2021-02-23 15:03 ./etc/
drwxr-xr-x root/root         0 2021-02-23 15:03 ./etc/phoenix/
drwxr-xr-x root/root         0 2021-02-23 15:03 ./etc/phoenix/conf.dist/
drwxr-xr-x root/root         0 2021-02-23 15:03 ./usr/
drwxr-xr-x root/root         0 2021-02-23 15:03 ./usr/lib/
drwxr-xr-x root/root         0 2021-02-23 15:03 ./usr/lib/phoenix/
drwxr-xr-x root/root         0 2021-02-23 15:03 ./usr/lib/phoenix/bin/
drwxr-xr-x root/root         0 2021-02-23 15:03 ./usr/lib/phoenix/bin/argparse-1.4.0/
-rw-r--r-- root/root     89214 2021-02-23 15:03 ./usr/lib/phoenix/bin/argparse-1.4.0/argparse.py
-rwxr-xr-x root/root     32864 2021-02-23 15:03 ./usr/lib/phoenix/bin/daemon.py
-rwxr-xr-x root/root      1899 2021-02-23 15:03 ./usr/lib/phoenix/bin/end2endTest.py
-rw-r--r-- root/root      1621 2021-02-23 15:03 ./usr/lib/phoenix/bin/hadoop-metrics2-hbase.properties
-rw-r--r-- root/root      3056 2021-02-23 15:03 ./usr/lib/phoenix/bin/hadoop-metrics2-phoenix.properties
-rw-r--r-- root/root      1886 2021-02-23 15:03 ./usr/lib/phoenix/bin/hbase-omid-client-config.yml
-rw-r--r-- root/root      1084 2021-02-23 15:03 ./usr/lib/phoenix/bin/hbase-site.xml
-rw-r--r-- root/root      2583 2021-02-23 15:03 ./usr/lib/phoenix/bin/log4j.properties
-rwxr-xr-x root/root      5194 2021-02-23 15:03 ./usr/lib/phoenix/bin/performance.py
-rwxr-xr-x root/root      2804 2021-02-23 15:03 ./usr/lib/phoenix/bin/pherf-standalone.py
-rwxr-xr-x root/root      9037 2021-02-23 15:03 ./usr/lib/phoenix/bin/phoenix_utils.py
-rwxr-xr-x root/root      2814 2021-02-23 15:03 ./usr/lib/phoenix/bin/psql.py
-rw-r--r-- root/root      1823 2021-02-23 15:03 ./usr/lib/phoenix/bin/readme.txt
-rwxr-xr-x root/root      4646 2021-02-23 15:03 ./usr/lib/phoenix/bin/sqlline.py
-rwxr-xr-x root/root      7226 2021-02-23 15:03 ./usr/lib/phoenix/bin/traceserver.py
-rw-r--r-- root/root 152051939 2021-02-23 15:03 ./usr/lib/phoenix/phoenix-client-hbase-2.2-5.1.0.jar
drwxr-xr-x root/root         0 2021-02-23 15:03 ./usr/share/
drwxr-xr-x root/root         0 2021-02-23 15:03 ./usr/share/doc/
drwxr-xr-x root/root         0 2021-02-23 15:03 ./usr/share/doc/phoenix/
-rw-r--r-- root/root     33895 2020-01-22 15:10 ./usr/share/doc/phoenix/LICENSE.gz
-rw-r--r-- root/root      3325 2020-01-22 15:10 ./usr/share/doc/phoenix/NOTICE.gz
drwxr-xr-x root/root         0 2020-01-22 15:10 ./usr/share/doc/phoenix/examples/
-rw-r--r-- root/root       149 2020-01-22 15:10 ./usr/share/doc/phoenix/examples/STOCK_SYMBOL.csv
-rw-r--r-- root/root       183 2020-01-22 15:10 ./usr/share/doc/phoenix/examples/STOCK_SYMBOL.sql
-rw-r--r-- root/root      2091 2020-01-22 15:10 ./usr/share/doc/phoenix/examples/WEB_STAT.csv
-rw-r--r-- root/root       296 2020-01-22 15:10 ./usr/share/doc/phoenix/examples/WEB_STAT.sql
-rw-r--r-- root/root       563 2020-01-22 15:10 ./usr/share/doc/phoenix/examples/WEB_STAT_QUERIES.sql
drwxr-xr-x root/root         0 2021-02-23 15:03 ./usr/share/doc/phoenix-client/
-rw-r--r-- root/root       131 2021-02-23 15:03 ./usr/share/doc/phoenix-client/changelog.Debian.gz
-rw-r--r-- root/root     24257 2021-02-05 08:27 ./usr/share/doc/phoenix-client/changelog.gz
-rw-r--r-- root/root       318 2021-02-23 15:03 ./usr/share/doc/phoenix-client/copyright
lrwxrwxrwx root/root         0 2021-02-23 15:03 ./usr/lib/phoenix/phoenix-client.jar -> phoenix-client-hbase-2.2-5.1.0.jar
$ cd provisioner/docker
$ ./docker-hadoop.sh -d -k hdfs,hbase -L -C config_debian-10.yaml -c 3

...

$ for i in $(seq 1 3); do
>   if [ $i -eq 1 ]; then ./docker-hadoop.sh --exec $i apt-get install -y phoenix-client; fi
>   ./docker-hadoop.sh --exec $i apt-get install -y phoenix
>   ./docker-hadoop.sh --exec $i ln -s /usr/lib/phoenix/phoenix-server.jar /usr/lib/hbase/lib/
>   ./docker-hadoop.sh --exec $i systemctl restart hbase-regionserver
>   if [ $i -eq 1 ]; then ./docker-hadoop.sh --exec $i systemctl restart hbase-master; fi
> done

...

$ ./docker-hadoop.sh --exec 1 python3 /usr/lib/phoenix/bin/sqlline.py

...

Connected to: Phoenix (version 5.1)
Driver: PhoenixEmbeddedDriver (version 5.1)
Autocommit status: true
Transaction isolation: TRANSACTION_READ_COMMITTED
sqlline version 1.9.0
0: jdbc:phoenix:> CREATE TABLE t(c VARCHAR PRIMARY KEY);
No rows affected (2.476 seconds)
0: jdbc:phoenix:> UPSERT INTO t VALUES ('foo');
1 row affected (0.138 seconds)
0: jdbc:phoenix:> SELECT * FROM t;
+-----+
|  C  |
+-----+
| foo |
+-----+
1 row selected (0.072 seconds)

sekikn avatar Feb 23 '21 23:02 sekikn

Thanks for working on this, @sekikn.

ln -s /usr/lib/phoenix/phoenix-server.jar /usr/lib/hbase/lib/

Should this symbolic link be contained in the phoenix package? Creating the symbolic link by scriptlet could be an another option.

[root@b0e6cfbecc83 /]# /usr/lib/phoenix/bin/sqlline.py
env: ‘python’: No such file or directory

I got this error on CentOS 8 bacause python2/python3 was not installed. The phoenix-client RPM should requires python3?

iwasakims avatar Feb 24 '21 12:02 iwasakims 

$ rpm -qlp output/phoenix/noarch/phoenix-5.1.0-1.el7.noarch.rpm 
/etc/default/phoenix
/usr/lib/phoenix/phoenix-pherf-5.1.0.jar
/usr/lib/phoenix/phoenix-server-hbase-2.2-5.1.0.jar
/usr/lib/phoenix/phoenix-server.jar
  • Should this package be phoenix-server rather than phoenix?
  • There seems to be no init script using the /etc/default/phoenix.

iwasakims avatar Feb 24 '21 13:02 iwasakims 

$ rpm -qlp output/phoenix/noarch/phoenix-client-5.1.0-1.el7.noarch.rpm 
/etc/phoenix/conf.dist
...
/usr/lib/phoenix/bin/traceserver.py
  • /etc/phoenix/conf.dist seems not to be used.
  • traceserver.py looks not client-side module. It will not work because relevant Java classes are not contained in phoenix-client.jar (nor phoenix-server.jar).

iwasakims avatar Feb 24 '21 13:02 iwasakims