bigtop icon indicating copy to clipboard operation
bigtop copied to clipboard

BIGTOP-3840. Enable short circuit local read in HDFS deployment by default.

Open iwasakims opened this issue 2 years ago • 1 comments

https://issues.apache.org/jira/browse/BIGTOP-3840

Short circuit local read is important feature for data store like HBase. It should be enabled by Puppet provisioner for testing the feature of libhadoop.so.

iwasakims avatar Oct 12 '22 05:10 iwasakims

Tested this by running smoke-tests of hdfs, yarn, mapreduce, hbase on Rocky Linux 8 x86_64 and Ubuntu 18.04 aarch64.

iwasakims avatar Oct 13 '22 02:10 iwasakims

+1, tested on CentOS 7.

$ curl -sL https://github.com/apache/bigtop/pull/1033.diff | git apply 
$ ./gradlew allclean hadoop-pkg repo -Dbuildwithdeps=true
$ cd provisioner/docker
$ ./docker-hadoop.sh -d -L -k hdfs -c 1
$ ./docker-hadoop.sh -e 1 grep -C2 shortcircuit /etc/hadoop/conf/hdfs-site.xml

  <property>
    <name>dfs.client.read.shortcircuit</name>
    <value>true</value>
  </property>

sekikn avatar Nov 21 '22 13:11 sekikn

I merged this. Thanks, @sekikn.

iwasakims avatar Nov 21 '22 14:11 iwasakims