ratis RATIS-1669. Combine shell lib folder and root jars folder

RATIS-1669. Combine shell lib folder and root jars folder

Open xleoken opened this issue 1 year ago • 4 comments

What changes were proposed in this pull request?

The shell jar in the lib folder, while other jars in jars folder, we can put them together.

[dcadmin@dcadmin-work ratis]$ ll -a ratis-assembly/target/apache-ratis-3.0.0-SNAPSHOT
total 36
drwxr-xr-x 11 dcadmin dcadmin   178 Jan 22  2020 .
drwxrwxr-x  7 dcadmin dcadmin   264 Aug  8 11:31 ..
drwxr-xr-x  2 dcadmin dcadmin    19 Jan 22  2020 bin
drwxr-xr-x  2 dcadmin dcadmin    30 Jan 22  2020 conf
drwxr-xr-x  2 dcadmin dcadmin  4096 Jan 22  2020 jars
drwxr-xr-x  3 dcadmin dcadmin    19 Jan 22  2020 lib
drwxr-xr-x  2 dcadmin dcadmin    35 Jan 22  2020 libexec
-rw-r--r--  1 dcadmin dcadmin 13394 Jan 22  2020 LICENSE
-rw-r--r--  1 dcadmin dcadmin  5246 Jan 22  2020 NOTICE
-rw-r--r--  1 dcadmin dcadmin  1201 Jan 22  2020 README.md
drwxr-xr-x 21 dcadmin dcadmin  4096 Jan 22  2020 src

After patch

[dcadmin@dcadmin-work apache-ratis-3.0.0-SNAPSHOT-bin]$ ll
total 32
drwxr-xr-x 2 dcadmin dcadmin    19 Jan 22  2020 bin
drwxr-xr-x 2 dcadmin dcadmin    30 Jan 22  2020 conf
drwxrwxr-x 5 dcadmin dcadmin    40 Aug 10 19:40 examples
drwxrwxr-x 2 dcadmin dcadmin  4096 Aug 10 19:40 jars
drwxr-xr-x 2 dcadmin dcadmin    35 Jan 22  2020 libexec
-rw-r--r-- 1 dcadmin dcadmin 13394 Jan 22  2020 LICENSE
drwxrwxr-x 2 dcadmin dcadmin    29 Aug 10 19:41 logs
-rw-r--r-- 1 dcadmin dcadmin  5537 Jan 22  2020 NOTICE
-rw-r--r-- 1 dcadmin dcadmin  1201 Jan 22  2020 README.md


[dcadmin@dcadmin-work apache-ratis-3.0.0-SNAPSHOT-bin]$ ll jars
total 24064
-rw-r--r-- 1 dcadmin dcadmin    58284 Jan 22  2020 commons-cli-1.5.0.jar
-rw-r--r-- 1 dcadmin dcadmin   851531 Jan 22  2020 javassist-3.28.0-GA.jar
-rw-r--r-- 1 dcadmin dcadmin    19936 Jan 22  2020 jsr305-3.0.2.jar
-rw-r--r-- 1 dcadmin dcadmin   489884 Jan 22  2020 log4j-1.2.17.jar
-rw-r--r-- 1 dcadmin dcadmin   116713 Jan 22  2020 ratis-client-3.0.0-SNAPSHOT.jar
-rw-r--r-- 1 dcadmin dcadmin   333006 Jan 22  2020 ratis-common-3.0.0-SNAPSHOT.jar
-rw-r--r-- 1 dcadmin dcadmin   126676 Jan 22  2020 ratis-grpc-3.0.0-SNAPSHOT.jar
-rw-r--r-- 1 dcadmin dcadmin    27006 Jan 22  2020 ratis-metrics-3.0.0-SNAPSHOT.jar
-rw-r--r-- 1 dcadmin dcadmin   111495 Jan 22  2020 ratis-netty-3.0.0-SNAPSHOT.jar
-rw-r--r-- 1 dcadmin dcadmin  1419397 Jan 22  2020 ratis-proto-3.0.0-SNAPSHOT.jar
-rw-r--r-- 1 dcadmin dcadmin     7266 Jan 22  2020 ratis-replicated-map-3.0.0-SNAPSHOT.jar
-rw-r--r-- 1 dcadmin dcadmin    46664 Jan 22  2020 ratis-resource-bundle-3.0.0-SNAPSHOT.jar
-rw-r--r-- 1 dcadmin dcadmin   416433 Jan 22  2020 ratis-server-3.0.0-SNAPSHOT.jar
-rw-r--r-- 1 dcadmin dcadmin    90772 Jan 22  2020 ratis-server-api-3.0.0-SNAPSHOT.jar
-rw-r--r-- 1 dcadmin dcadmin  1602623 Jan 22  2020 ratis-shell-3.0.0-SNAPSHOT-jar-with-dependencies.jar
-rw-r--r-- 1 dcadmin dcadmin     6826 Jan 22  2020 ratis-test-3.0.0-SNAPSHOT.jar
-rw-r--r-- 1 dcadmin dcadmin 18685030 Jan 22  2020 ratis-thirdparty-misc-1.0.1.jar
-rw-r--r-- 1 dcadmin dcadmin    11618 Jan 22  2020 ratis-tools-3.0.0-SNAPSHOT.jar
-rw-r--r-- 1 dcadmin dcadmin   130398 Jan 22  2020 reflections-0.10.2.jar
-rw-r--r-- 1 dcadmin dcadmin    41424 Jan 22  2020 slf4j-api-1.7.29.jar
-rw-r--r-- 1 dcadmin dcadmin    12211 Jan 22  2020 slf4j-log4j12-1.7.29.jar
[dcadmin@dcadmin-work apache-ratis-3.0.0-SNAPSHOT-bin]$

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/RATIS-1669

How was this patch tested?

run bin/ratis command.

[dcadmin@dcadmin-work apache-ratis-3.0.0-SNAPSHOT-bin]$ bin/ratis sh
Usage: ratis sh [generic options]
	 [election [transfer] [stepDown] [pause] [resume]]         
	 [group [info] [list]]                                     
	 [peer [add] [remove] [setPriority]]                       
	 [snapshot [create]]

Aug 10 '22 11:08 xleoken

hi @szetszwo, take a final check.

I forget to exclude the example from the pkg, but it seems doesn't have a good way to resolve it. Trying to think a best way to solve it.

Do we need to exclude it in this patch currently?

Aug 11 '22 02:08 xleoken

Do we need to exclude it in this patch currently?

Your choice. We may do it here or in the next change.

Aug 11 '22 05:08 szetszwo

hi @szetszwo, take a final check, keep the example in the pkg.

the latest pkg structure in local dev environment.

[dcadmin@dcadmin-work apache-ratis-3.0.0-SNAPSHOT-bin]$ du -chs *
4.0K	bin
4.0K	conf
21M	examples
23M	jars
4.0K	libexec
16K	LICENSE
8.0K	NOTICE
4.0K	README.md
44M	total

[dcadmin@dcadmin-work apache-ratis-3.0.0-SNAPSHOT-bin]$ ll
total 32
drwxr-xr-x 2 dcadmin dcadmin    19 Jan 22  2020 bin
drwxr-xr-x 2 dcadmin dcadmin    30 Jan 22  2020 conf
drwxrwxr-x 5 dcadmin dcadmin    40 Aug 11 13:48 examples
drwxrwxr-x 2 dcadmin dcadmin  4096 Aug 11 13:48 jars
drwxr-xr-x 2 dcadmin dcadmin    35 Jan 22  2020 libexec
-rw-r--r-- 1 dcadmin dcadmin 13419 Jan 22  2020 LICENSE
-rw-r--r-- 1 dcadmin dcadmin  5246 Jan 22  2020 NOTICE
-rw-r--r-- 1 dcadmin dcadmin  1201 Jan 22  2020 README.md

[dcadmin@dcadmin-work apache-ratis-3.0.0-SNAPSHOT-bin]$ ll bin
total 4
-rwxr-xr-x 1 dcadmin dcadmin 2152 Jan 22  2020 ratis

[dcadmin@dcadmin-work apache-ratis-3.0.0-SNAPSHOT-bin]$ ll examples/bin/
total 20
-rwxr-xr-x 1 dcadmin dcadmin 1138 Jan 22  2020 client.sh
-rwxr-xr-x 1 dcadmin dcadmin 1859 Jan 22  2020 common.sh
-rwxr-xr-x 1 dcadmin dcadmin  921 Jan 22  2020 server.sh
-rwxr-xr-x 1 dcadmin dcadmin 1598 Jan 22  2020 start-all.sh
-rwxr-xr-x 1 dcadmin dcadmin  922 Jan 22  2020 stop-all.sh

[dcadmin@dcadmin-work apache-ratis-3.0.0-SNAPSHOT-bin]$ ll jars/
total 23124
-rw-r--r-- 1 dcadmin dcadmin    58284 Jan 22  2020 commons-cli-1.5.0.jar
-rw-r--r-- 1 dcadmin dcadmin   851531 Jan 22  2020 javassist-3.28.0-GA.jar
-rw-r--r-- 1 dcadmin dcadmin    19936 Jan 22  2020 jsr305-3.0.2.jar
-rw-r--r-- 1 dcadmin dcadmin   116713 Jan 22  2020 ratis-client-3.0.0-SNAPSHOT.jar
-rw-r--r-- 1 dcadmin dcadmin   333476 Jan 22  2020 ratis-common-3.0.0-SNAPSHOT.jar
-rw-r--r-- 1 dcadmin dcadmin   127010 Jan 22  2020 ratis-grpc-3.0.0-SNAPSHOT.jar
-rw-r--r-- 1 dcadmin dcadmin    27006 Jan 22  2020 ratis-metrics-3.0.0-SNAPSHOT.jar
-rw-r--r-- 1 dcadmin dcadmin   111493 Jan 22  2020 ratis-netty-3.0.0-SNAPSHOT.jar
-rw-r--r-- 1 dcadmin dcadmin  1419395 Jan 22  2020 ratis-proto-3.0.0-SNAPSHOT.jar
-rw-r--r-- 1 dcadmin dcadmin     7264 Jan 22  2020 ratis-replicated-map-3.0.0-SNAPSHOT.jar
-rw-r--r-- 1 dcadmin dcadmin    46660 Jan 22  2020 ratis-resource-bundle-3.0.0-SNAPSHOT.jar
-rw-r--r-- 1 dcadmin dcadmin   417972 Jan 22  2020 ratis-server-3.0.0-SNAPSHOT.jar
-rw-r--r-- 1 dcadmin dcadmin    90832 Jan 22  2020 ratis-server-api-3.0.0-SNAPSHOT.jar
-rw-r--r-- 1 dcadmin dcadmin  1119277 Jan 22  2020 ratis-shell-3.0.0-SNAPSHOT.jar
-rw-r--r-- 1 dcadmin dcadmin     6825 Jan 22  2020 ratis-test-3.0.0-SNAPSHOT.jar
-rw-r--r-- 1 dcadmin dcadmin 18685030 Jan 22  2020 ratis-thirdparty-misc-1.0.1.jar
-rw-r--r-- 1 dcadmin dcadmin    11615 Jan 22  2020 ratis-tools-3.0.0-SNAPSHOT.jar
-rw-r--r-- 1 dcadmin dcadmin   130398 Jan 22  2020 reflections-0.10.2.jar
-rw-r--r-- 1 dcadmin dcadmin    41424 Jan 22  2020 slf4j-api-1.7.29.jar
-rw-r--r-- 1 dcadmin dcadmin    15240 Jan 22  2020 slf4j-simple-1.7.29.jar

[dcadmin@dcadmin-work apache-ratis-3.0.0-SNAPSHOT-bin]$ bin/ratis sh
[main] INFO org.reflections.Reflections - Reflections took 57 ms to scan 1 urls, producing 5 keys and 18 values
Usage: ratis sh [generic options]
	 [election [transfer] [stepDown] [pause] [resume]]         
	 [group [info] [list]]                                     
	 [peer [add] [remove] [setPriority]]                       
	 [snapshot [create]]

Aug 11 '22 05:08 xleoken

@leo65535 , thanks for the update! There are some classes in ratis-shell-3.0.0-SNAPSHOT.jar duplicated with the classes in other jars.

Nice catch, I had remove the shade plugin from the shell module, the patch has been updated, thanks. Also checked the bin/ratis command, everything is ok.

Aug 12 '22 01:08 xleoken

Nice, will ask someone else check the permission issuce again.

Aug 12 '22 09:08 xleoken

@leo65535 , FYI, the scripts in our 2.3.0 release also do not have execute permission after untar:

$ls -l apache-ratis-2.3.0/dev-support/checks
total 72
-rw-r--r--@ 1 szetszwo  staff  3396 Jan 22  2020 _mvn_unit_report.sh
-rw-r--r--@ 1 szetszwo  staff  1189 Jan 22  2020 author.sh
-rw-r--r--@ 1 szetszwo  staff  1000 Jan 22  2020 build.sh
-rw-r--r--@ 1 szetszwo  staff  1937 Jan 22  2020 checkstyle.sh
-rw-r--r--@ 1 szetszwo  staff  1695 Jan 22  2020 findbugs.sh
-rw-r--r--@ 1 szetszwo  staff  1268 Jan 22  2020 rat.sh
-rw-r--r--@ 1 szetszwo  staff  1384 Jan 22  2020 shellcheck.sh
-rw-r--r--@ 1 szetszwo  staff  1197 Jan 22  2020 sonar.sh
-rw-r--r--@ 1 szetszwo  staff  1249 Jan 22  2020 unit.sh

In the project dir, the original scripts do have execute permission

$ls -l dev-support/checks
total 72
-rwxr-xr-x  1 szetszwo  staff  3396 May 19 08:14 _mvn_unit_report.sh
-rwxr-xr-x  1 szetszwo  staff  1189 May 19 08:14 author.sh
-rwxr-xr-x  1 szetszwo  staff  1005 May 19 08:14 build.sh
-rwxr-xr-x  1 szetszwo  staff  1937 May 19 08:14 checkstyle.sh
-rwxr-xr-x  1 szetszwo  staff  1695 May 19 08:14 findbugs.sh
-rwxr-xr-x  1 szetszwo  staff  1268 May 19 08:14 rat.sh
-rwxr-xr-x  1 szetszwo  staff  1384 May 19 08:14 shellcheck.sh
-rwxr-xr-x  1 szetszwo  staff  1197 May 19 08:14 sonar.sh
-rwxr-xr-x  1 szetszwo  staff  1249 May 19 08:14 unit.sh

Aug 12 '22 17:08 szetszwo

The script files inside apache-ratis-2.3.0-bin.tar.gz also do not have execute permission

$tar tvf apache-ratis-2.3.0-bin.tar.gz  | grep sh
-rw-r--r--  0 root   root     3396 Jan 22  2020 apache-ratis-2.3.0/dev-support/checks/_mvn_unit_report.sh
-rw-r--r--  0 root   root     1189 Jan 22  2020 apache-ratis-2.3.0/dev-support/checks/author.sh
-rw-r--r--  0 root   root     1000 Jan 22  2020 apache-ratis-2.3.0/dev-support/checks/build.sh
-rw-r--r--  0 root   root     1937 Jan 22  2020 apache-ratis-2.3.0/dev-support/checks/checkstyle.sh
-rw-r--r--  0 root   root     1695 Jan 22  2020 apache-ratis-2.3.0/dev-support/checks/findbugs.sh
-rw-r--r--  0 root   root     1268 Jan 22  2020 apache-ratis-2.3.0/dev-support/checks/rat.sh
-rw-r--r--  0 root   root     1384 Jan 22  2020 apache-ratis-2.3.0/dev-support/checks/shellcheck.sh
-rw-r--r--  0 root   root     1197 Jan 22  2020 apache-ratis-2.3.0/dev-support/checks/sonar.sh
-rw-r--r--  0 root   root     1249 Jan 22  2020 apache-ratis-2.3.0/dev-support/checks/unit.sh
-rw-r--r--  0 root   root     1690 Jan 22  2020 apache-ratis-2.3.0/dev-support/ci/common.sh
-rw-r--r--  0 root   root     2103 Jan 22  2020 apache-ratis-2.3.0/dev-support/ci/nightly-build.sh
-rw-r--r--  0 root   root     2113 Jan 22  2020 apache-ratis-2.3.0/dev-support/ci/precommit-build.sh
-rw-r--r--  0 root   root     1091 Jan 22  2020 apache-ratis-2.3.0/dev-support/intellij/install-runconfig.sh
-rw-r--r--  0 root   root     6319 Jan 22  2020 apache-ratis-2.3.0/dev-support/make_rc.sh
-rw-r--r--  0 root   root     1382 Jan 22  2020 apache-ratis-2.3.0/dev-support/run-test-repeatedly.sh
-rw-r--r--  0 root   root     1345 Jan 22  2020 apache-ratis-2.3.0/dev-support/vagrant/bin/start_ratis_load_gen.sh
-rw-r--r--  0 root   root     1066 Jan 22  2020 apache-ratis-2.3.0/dev-support/vagrant/bin/start_ratis_server.sh
-rw-r--r--  0 root   root     2509 Jan 22  2020 apache-ratis-2.3.0/dev-support/vagrant/run_all_tests.sh
-rw-r--r--  0 root   root     1594 Jan 22  2020 apache-ratis-2.3.0/dev-support/yetus-personality.sh

Aug 12 '22 17:08 szetszwo

ratis ratis copied to clipboard

RATIS-1669. Combine shell lib folder and root jars folder

What changes were proposed in this pull request?

What is the link to the Apache JIRA

How was this patch tested?

ratis
ratis copied to clipboard