shini
shini copied to clipboard
Feature request - only read specific sections of ini file
Hi. First thank you for providing shini it is working fine.
I would like to request a new feature, because if we have a long ini file it takes a long time to read, and we often only need to read a specific section of the ini file. So could you add the possibility to only read the variables in a specified section?
Thanks
Absolutely - if this project is seeing this much attention, I'll happily spend the time trying to optimise it!
As soon as possible, I'll investigate general optimisations (to increase the speed overall) and, as you suggest, a specific enhancement to deal with only the section in question.
Thank you very much.
We have included your "Shini" script in our small free audio player for Raspberry - the piCorePlayer: https://sites.google.com/site/picoreplayer/home
That's really cool :-) May I ask for an example copy of the INI file you're parsing, and an example of the operation(s) you're performing on it?
I've made some fairly aggressive changes for bash
>= version 3 and zsh
(I'm not clear on the version compatibility for zsh
), which has pushed the performance to almost 7x faster (in my tests).
Regarding processing a specific section, am I understanding correctly that you'd like to be able to specify shini_parse ... "SOMEBLOCK"
so it only parses the content of that specific section?
I've included PHP's shipped php.ini
as a performance test, and included test.sh
which accepts a single argument for the specific INI section to parse. (It simply skips everything else.) Not specifying the argument parses the entire file as normal.
The overall performance increase is very significant with the earlier changes:
Before
time bash test_perf.sh
real 0m16.372s
After
time bash test_perf.sh
real 0m0.789s
(Note, zsh
seems to have the edge on performance in the current implementation.)
Including specifying the section, further gains can be made:
time bash ./test_perf.sh > /dev/null
real 0m0.838s
time bash ./test_perf.sh PHP > /dev/null
real 0m0.396s
time bash ./test_perf.sh opcache > /dev/null
real 0m0.309s
time bash ./test_perf.sh curl > /dev/null
real 0m0.282s
This is great :-) Here is our script that among many things also use shini to read a lot of audio DAC.conf files in order to make us able to use the variables in the DACs
#!/bin/sh
# Version: 3.20 2017-03-12
# Added check for /tmp/dropdown.cfg in pcp_sound_card_dropdown. SBP.
# Added shini conf parser. SBP.
# Suppress already mounted message during do_rebootstuff newconfig process. PH.
# Version: 3.11 2017-01-20
# Fixed the issue with shairport not working when using USB cards. SBP.
# Version: 3.10 2016-12-11
# Original. SBP.
pcp_debug_log "$0 - Running pcp-soundcard-functions..."
DACLOCATION="/usr/local/etc/pcp/cards" # <=== Should be in pcp-functions? GE.
#The below line is for development only.
#DACLOCATION="/home/tc/www/cgi-bin/cards"
############
. /usr/local/bin/shini.sh
#========================================================================================
# Check if we use the special Audio kernel or not. As we load special overlays if using Audio kernel
#----------------------------------------------------------------------------------------
pcp_AudioKernel_status() {
sudo uname -a | grep pcpAudioCore > /dev/null 2>&1 && KERNELVERSION=Audiokernel || KERNELVERSION=Officialkernel
}
pcp_AudioKernel_status
#========================================================================================
# Disable/re_enable analog audio
#----------------------------------------------------------------------------------------
pcp_disable_analog() {
sudo sed -i 's/^dtparam=audio=on/#dtparam=audio=on/g' $CONFIGTXT
sudo sed -i 's/^audio_pwm_mode=2/#audio_pwm_mode=2/g' $CONFIGTXT
}
pcp_re_enable_analog() {
pcp_disable_analog
sudo sed -i "/dtparam=audio=on/c\dtparam=audio=on" $CONFIGTXT
sudo sed -i "/audio_pwm_mode=2/c\audio_pwm_mode=2" $CONFIGTXT
}
#========================================================================================
# Remove Audio dtoverlays
#----------------------------------------------------------------------------------------
pcp_disable_i2s() {
pcp_re_enable_analog
# unset DTOVERLAY both the old one that is removed on chooseoutput.cgi and the one from config.cfg to clear all traces of audio dtoverlays
shini_parse ${DACLOCATION}/${ORIG_AUDIO}.conf
[ x"" != x"${DTOVERLAY}" ] && sed -i '/dtoverlay='"$DTOVERLAY"'/d' $CONFIGTXT
shini_parse ${DACLOCATION}/${AUDIO}.conf
[ x"" != x"${DTOVERLAY}" ] && sed -i '/dtoverlay='"$DTOVERLAY"'/d' $CONFIGTXT
}
#========================================================================================
# Add Audio dtoverlays
#----------------------------------------------------------------------------------------
pcp_enable_i2s() {
pcp_re_enable_analog
pcp_selected_soundcontrol
[ x"$SPARAMS1" != x"" ] && PAR1=','"$SPARAMS1"'' || PAR1="$SPARAMS1"
[ x"$SPARAMS2" != x"" ] && PAR2=','"$SPARAMS2"'' || PAR2="$SPARAMS2"
[ x"$SPARAMS3" != x"" ] && PAR3=','"$SPARAMS3"'' || PAR3="$SPARAMS3"
[ x"$SPARAMS4" != x"" ] && PAR4=','"$SPARAMS4"'' || PAR4="$SPARAMS4"
[ x"$SPARAMS5" != x"" ] && PAR5=','"$SPARAMS5"'' || PAR5="$SPARAMS5"
[ x"$DTOVERLAY" != x"" ] && sudo echo dtoverlay=$DTOVERLAY$PAR1$PAR2$PAR3$PAR4$PAR5 >> $CONFIGTXT
}
#========================================================================================
# Enable/disable HDMI settings in config.txt
#----------------------------------------------------------------------------------------
pcp_disable_HDMI() {
sed -i '/hdmi_drive=2/d' $CONFIGTXT
sed -i '/hdmi_force_hotplug=1/d' $CONFIGTXT
sed -i '/hdmi_force_edid_audio=1/d' $CONFIGTXT
}
pcp_enable_HDMI() {
pcp_disable_i2s
sudo echo hdmi_drive=2 >> $CONFIGTXT
sudo echo hdmi_force_hotplug=1 >> $CONFIGTXT
sudo echo hdmi_force_edid_audio=1 >> $CONFIGTXT
sudo amixer cset numid=3 2 >/dev/null 2>&1
}
#========================================================================================
# Section that controls loading of DAC overlays
#----------------------------------------------------------------------------------------
pcp_read_chosen_audio() {
# if noumount, disk is probably already mounted....supress output
[ "$1" == "noumount" ] && pcp_mount_mmcblk0p1 > /dev/null 2>&1 || pcp_mount_mmcblk0p1
pcp_disable_HDMI
pcp_disable_i2s
pcp_enable_i2s
# This is to allow no unmount of mmcblk0p1 during booting with newconfig.cfg
[ "$1" != "noumount" ] && pcp_umount_mmcblk0p1
}
#========================================================================================
# Control of sound card routines:
# - Needs to be populated for each type of soundcard.
# - Some are without filter options.
# - USB cards will probably never be supported this way as they are so different.
#----------------------------------------------------------------------------------------
# HELP - To find the controls:
# - amixer -c 0 scontrols
# - aplay -l
#----------------------------------------------------------------------------------------
pcp_generic_card_control() {
case "$GENERIC_CARD" in
TI51XX)
amixer -c $CARD scontrols | grep -q 'DSP Program' && DSPfound="yes" || DSPfound="no"
if [ "$DSPfound" = "yes" ]; then
DSP="DSP Program,0"
FILTER1="Low latency IIR with de-emphasis"
FILTER2="FIR interpolation with de-emphasis"
FILTER3="High attenuation with de-emphasis"
FILTER4="Fixed process flow"
FILTER5="Ringing-less low latency FIR"
fi
ACTUAL_VOL=$(amixer -c $CARD sget $SSET | grep "Right: Playback" | awk '{ print $5 }' | tr -d "[]%")
ACTUAL_DB=$(amixer -c $CARD sget $SSET | grep "Right: Playback" | awk '{ print $6 }' | tr -d "[]")
ACTUAL_FILTER=$(amixer -c $CARD sget 'DSP Program,0' | grep "Item0:" | awk '{ print $2 }' | tr -d "'")
case "$DSPFILTER" in
FILTER1) FILTER="$FILTER1" ;;
FILTER2) FILTER="$FILTER2" ;;
FILTER3) FILTER="$FILTER3" ;;
FILTER4) FILTER="$FILTER4" ;;
FILTER5) FILTER="$FILTER5" ;;
esac
# Logic to make checked radiobuttons - needs to clear it otherwise we have two FILTERS_CHECK checked.
FILTER1_CHECK=""
FILTER2_CHECK=""
FILTER3_CHECK=""
FILTER4_CHECK=""
FILTER5_CHECK=""
case "$ACTUAL_FILTER" in
Low) FILTER1_CHECK="checked" ;;
FIR) FILTER2_CHECK="checked" ;;
High) FILTER3_CHECK="checked" ;;
Fixed) FILTER4_CHECK="checked" ;;
Ringing-less) FILTER5_CHECK="checked" ;;
esac
;;
HIFIBERRY_AMP)
ACTUAL_VOL=$(amixer -c $CARD sget $SSET | grep "Mono:" | awk '{ print $3 }' | tr -d "[]%")
;;
ONBOARD)
ACTUAL_VOL=$(amixer -c $CARD sget $SSET | grep "Mono: Playback" | awk '{ print $4 }' | tr -d "[]%")
ACTUAL_DB=$(amixer -c $CARD sget $SSET | grep "Mono: Playback" | awk '{ print $5 }' | tr -d "[]")
;;
# ES9023)
# CARD="ALSA"
# SSET="PCM"
# ACTUAL_VOL=$(amixer -c $CARD sget $SSET | grep "Mono: Playback" | awk '{ print $4 }' | tr -d "[]%")
# ACTUAL_DB=$(amixer -c $CARD sget $SSET | grep "Mono: Playback" | awk '{ print $5 }' | tr -d "[]")
# TEXT="Choosing 384k will enable 352k and 384k sample rates. Selecting bclk_ratio_int_div will use bclk_ratio=50 for 16/24bps and bclk_ratio=100 for 32bps media when sample rate is a multiple of 8kHz and less than 192kHz. Which causes the selection of the 19M2 OSC as the parent for the PCM clock with an integer divider, rather than PLLD with fractional divider and MASH noise shaping."
# PARAMS1="384k"
# PARAMS2="bclk_ratio_int_div"
# ;;
esac
}
#========================================================================================
# This will get us all available AUDIO DACs
#----------------------------------------------------------------------------------------
AUDIO_OPTIONS_EXTRA=$(ls "$DACLOCATION" | grep ".conf" | awk -F'.conf' '{ print $1 }')
#========================================================================================
# Section that populate sound card drop-down list on Squeezelite.cgi
#----------------------------------------------------------------------------------------
pcp_sound_card_dropdown() {
pcp_blank_audio_conf
. $CONFIGCFG
if [ -e /tmp/dropdown.cfg ]; then
sudo sed -i 's/:selected:/:notselected:/' /tmp/dropdown.cfg
sudo sed -i "/${AUDIO}:/s/:notselected:/:selected:/" /tmp/dropdown.cfg
else
CONF_AUDIO="$AUDIO"
pcp_blank_audio_conf
for i in $AUDIO_OPTIONS_EXTRA; do
AUDIO=$i
pcp_soundcontrol
CHECKED="notselected"
[ "$AUDIO" = "$CONF_AUDIO" ] && CHECKED="selected"
if [ x"$LISTNAME" != x ]; then
sudo echo "${AUDIO}:${CHECKED}:${LISTNAME}:${RPI_MODEL}" >> /tmp/dropdown.cfg
fi
done
fi
}
#========================================================================================
# This will get us available options from the DAC
#----------------------------------------------------------------------------------------
old_pcp_soundcontrol() {
pcp_blank_audio_conf
echo "$AUDIO_OPTIONS_EXTRA" | grep -q "$AUDIO" && . "$DACLOCATION"/"$AUDIO".conf
}
# Declare a handler for parsed variables. This is required.
__shini_parsed() {
# printf " %s_%s='%s'\n" "$1" "$2" "$3"
eval \ $1_$2='"$3"'
if [ "$KERNELVERSION" = "Audiokernel" ]; then
if [ "$1" = "PCPAUDIOCORE" ]; then
# printf " %s_%s='%s'\n" "$1" "$2" "$3"
eval \ $2='"$3"'
fi
fi
if [ "$KERNELVERSION" = "Officialkernel" ]; then
if [ "$1" = "PCPCORE" ]; then
# printf " %s_%s='%s'\n" "$1" "$2" "$3"
eval \ $2='"$3"'
fi
fi
if [ "$1" = "COMMON" ]; then
# printf " %s_%s='%s'\n" "$1" "$2" "$3"
eval \ $2='"$3"'
fi
}
pcp_soundcontrol() {
# Parse
shini_parse ${DACLOCATION}/${AUDIO}.conf
}
#========================================================================================
# This will get us available options from the selected DAC
#----------------------------------------------------------------------------------------
pcp_selected_soundcontrol() {
pcp_blank_audio_conf
. "$CONFIGCFG"
shini_parse ${DACLOCATION}/${AUDIO}.conf
}
pcp_blank_audio_conf() {
# Remove left-over from previous audio DAC
CARD=""
OUTPUT=""
ALSA_PARAMS=""
GENERIC_CARD=""
CONTROL_PAGE=""
LISTNAME=""
DTOVERLAY=""
SSET=""
PARAMS1=""
PARAMS2=""
PARAMS3=""
PARAMS4=""
PARAMS5=""
RPI_MODEL=""
I=1
while true; do
eval TEST="\${TEXT${I}}"
if [ x"$TEST" != x ]; then
eval TEXT${I}=""
I=$((I+1))
else
break
fi
done
}
Here is a sample DAC.conf file
[COMMON]
CARD="sndrpihifiberry"
LISTNAME="HiFiBerry DAC+ Light"
OUTPUT="hw:CARD=sndrpihifiberry"
ALSA_PARAMS="80:4::1"
RPI_MODEL="HAT_ALL"
[PCPAUDIOCORE]
GENERIC_CARD="ES9023"
DTOVERLAY="hifiberry-dacpluslight-es9023-audio"
TEXT1="<b>384k</b> will enable 352k and 384k sample rates."
TEXT2="<b>bclk_ratio_int_div</b> will use bclk_ratio=50 for 16/24bps and bclk_ratio=100 for 32bps media when sample rate is a multiple of 8kHz and less than 192kHz. Which causes the selection of the 19M2 OSC as the parent for the PCM clock with an integer divider, rather than PLLD with fractional divider and MASH noise shaping."
PARAMS1=""
PARAMS2="384k"
PARAMS3="bclk_ratio_int_div"
CONTROL_PAGE="soundcard_control.cgi"
[PCPCORE]
DTOVERLAY="hifiberry-dac"
CONTROL_PAGE=""
Please note that we are using ash #!/bin/sh as we are building on piCore which is using busybox
Yes we would like to be able to only parse a specific part of the ini file (for other parts of our player) - so your additions are highly appreciated.
Sorry - but the new shini script do not run on our system. This function stalls it all:
shini_regex_replace()
{
if [ "${BASH_VERSINFO}" -ge 3 ] || [ -n "$ZSH_VERSION" ]; then
[[ "$1" =~ $2 ]] && shini_retval=${BASH_REMATCH[1]} || shini_retval="$1"
return 0
fi
shini_retval="$(printf '%s' "$1" | sed -E "s/$2/\1/")" # If you have isses on older systems,
# it may be the non-newer POSIX compliant sed.
# -E should be enabling extended regex mode portably.
}
If I change it to this it runs, but ..:
shini_regex_replace()
{
# if [ "${BASH_VERSINFO}" -ge 3 ] || [ -n "$ZSH_VERSION" ]; then
# [[ "$1" =~ $2 ]] && shini_retval=${BASH_REMATCH[1]} || shini_retval="$1"
# return 0
# fi
shini_retval="$(printf '%s' "$1" | sed -E "s/$2/\1/")" # If you have isses on older systems,
# it may be the non-newer POSIX compliant sed.
# -E should be enabling extended regex mode portably.
}
Ah, I've not tested t under ash. Which version specifically are you using? I'll see if I can fix the incompatibility.
Ok. Could you let me know you shell version? (Specifically the shell version the script is being ran under.)
Sent from my iPhone
On 6 May 2017, at 13:43, amtssp [email protected] wrote:
Sorry - but the new shini script do wont run on our system. This function stalls it all:
shini_regex_replace() { if [ "${BASH_VERSINFO}" -ge 3 ] || [ -n "$ZSH_VERSION" ]; then [[ "$1" =~ $2 ]] && shini_retval=${BASH_REMATCH[1]} || shini_retval="$1" return 0 fi
shini_retval="$(printf '%s' "$1" | sed -E "s/$2/\1/")" # If you have isses on older systems, # it may be the non-newer POSIX compliant sed. # -E should be enabling extended regex mode portably.
} If I change it to this it runs, but ..:
shini_regex_replace() {
if [ "${BASH_VERSINFO}" -ge 3 ] || [ -n "$ZSH_VERSION" ]; then
[[ "$1" =~ $2 ]] && shini_retval=${BASH_REMATCH[1]} || shini_retval="$1"
return 0
fi
shini_retval="$(printf '%s' "$1" | sed -E "s/$2/\1/")" # If you have isses on older systems, # it may be the non-newer POSIX compliant sed. # -E should be enabling extended regex mode portably.
} — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.
Sorry for the delay. Is this what you were asking for?
tc@piCorePlayer:~$ ash --help BusyBox v1.24.2 (2016-09-08 12:09:43 UTC) multi-call binary.
Hi. Thanks for the changes now shini parse the file fine using busybox ash. It is still very slow (on a Raspberry pi Zerro Wireless) here is the result:
tc@piCorePlayer:~/www/cgi-bin$ time ash ./test_perf.sh > /dev/null
Command exited with non-zero status 1
real 2m 1.38s
user 0m 25.24s
sys 0m 52.27s
Do you think that you can improve the speed on ash as well? I noticed this discussion here: [http://stackoverflow.com/questions/21010882/how-to-match-regexp-with-ash]
I don't know if the last answer on that page will improve speed?
No problem, I realised there was a bug breaking the "fallback" functionality (which as you rightly notice, is extremely slow even on the best of hardware), which I've patched.
I too had read that article, but haven't had time to confirm it works yet. I'll give it a whirl this weekend and let you know. (I'm able to test locally against dash
, which I believe is a fork of ash
- and certainly seems to exhibit the same behaviour as you're seeing.)
I'll let you know ASAP. Thanks for your patience... :-)
Hi there. Is there anything I can do to help you in the process of making shini faster using busybox or ash?
I've had an extensive think about the problem and tried to debug where the duration is being spent.
I suspected - and I believe have now proved beyond doubt - that the slowness comes from calling grep
and sed
(which I'm using as a fallback for non bash
/zsh
, which both have regex match and replacement built in. ksh
has regex matching, but is relying on sed
for replacements.)
Thus the performance is significantly faster in zsh
and bash
, slower in ksh
and abysmal in anything else.
I'm not that familiar with ash
, so if you can think of any way to do what I'm achieving with those grep
and sed
calls in a more "builtin" manner, that'd help more than a lot.
I've spent some time trying to use named pipes, so there's only one grep
/sed
process spawned at the start for each of the regexes used - but as you can imagine it's super hacky (you need around 22 named pipes and 11 children processes to get anywhere close).
Otherwise I'm afraid I'm out of ideas, except to suggest switching to another library which you could call via anything else you have available (Python/PHP) or knock something up in C. (I'm happy to give that a go if you're willing to compile+ship it with your project. Bit of an ache though!)
There might be some further micro-optimisations around the order of execution etc, but I don't think it'll make enough of a difference for as long as grep
/sed
are being used.
FYI - these are the performance gains by using builtins from other shells. Note it is still sped up by searching a specific section rather than the entire file - but still awfully slow in dash
:
$ wc -l tests/php.ini
1917 tests/php.ini
$ time zsh ./test_perf.sh > /dev/null
real 0m0.595s
$ time bash ./test_perf.sh > /dev/null
real 0m0.838s
$ time ksh ./test_perf.sh > /dev/null
real 0m2.901s
$ time zsh ./test_perf.sh opcache > /dev/null
real 0m0.237s
$ time bash ./test_perf.sh opcache > /dev/null
real 0m0.313s
$ time ksh ./test_perf.sh opcache > /dev/null
real 0m0.543s
$ time dash test_perf.sh > /dev/null
real 0m17.930s
$ time dash test_perf.sh opcache > /dev/null
real 0m5.839s
Thanks for trying. I noticed this on the busybox web page: https://www.busybox.net/BusyBox.html
expr
expr EXPRESSION
Print the value of EXPRESSION to stdout
EXPRESSION may be:
ARG1 | ARG2 ARG1 if it is neither null nor 0, otherwise ARG2
ARG1 & ARG2 ARG1 if neither argument is null or 0, otherwise 0
ARG1 < ARG2 1 if ARG1 is less than ARG2, else 0. Similarly:
ARG1 <= ARG2
ARG1 = ARG2
ARG1 != ARG2
ARG1 >= ARG2
ARG1 > ARG2
ARG1 + ARG2 Sum of ARG1 and ARG2. Similarly:
ARG1 - ARG2
ARG1 * ARG2
ARG1 / ARG2
ARG1 % ARG2
STRING : REGEXP Anchored pattern match of REGEXP in STRING
match STRING REGEXP Same as STRING : REGEXP
substr STRING POS LENGTH Substring of STRING, POS counted from 1
index STRING CHARS Index in STRING where any CHARS is found, or 0
length STRING Length of STRING
quote TOKEN Interpret TOKEN as a string, even if
it is a keyword like 'match' or an
operator like '/'
(EXPRESSION) Value of EXPRESSION
Beware that many operators need to be escaped or quoted for shells. Comparisons are arithmetic if both ARGs are numbers, else lexicographical. Pattern matches return the string matched between \( and \) or null; if \( and \) are not used, they return the number of characters matched or 0.
So can the expr command be useful?
OK, with the greatest emphasis - please don't get overly excited because I've not extensively tested this (and haven't yet found a way to get myself easy access to a busybox install!)
But, if I've correctly implemented what you found, these are the new speeds on dash
/ash
(verses the 17s before):
$ time dash test.sh
test.sh: 145: test.sh: cannot open tests/nonexistent.ini: No such file
real 0m0.707s
$ time bash test.sh
./shini.sh: line 81: tests/nonexistent.ini: No such file or directory
real 0m0.187s
I'll commit and push the (experimental) change to a separate branch and post details here momentarily.
https://github.com/wallyhall/shini/compare/expr-experimental
Simple change, I had tried expr
before but obviously wasn't doing it right. The documentation you referenced appears to work correctly in dash
at least - so here's to hoping!
tc@piCorePlayer:~/www/cgi-bin$ time ash ./test_perf.sh > /dev/null
./test_perf.sh: ./shini.sh: line 59: syntax error: bad substitution
Command exited with non-zero status 2
real 0m 0.02s
user 0m 0.01s
sys 0m 0.00s
If I comment that section again like this:
shini_regex_replace()
{
# if [ -n "$BASH_VERSINFO" ] && [ "${BASH_VERSINFO}" -ge 3 ] || \
# [ -n "$ZSH_VERSION" ]; then
# [[ "$1" =~ $2 ]] && shini_retval=${BASH_REMATCH[1]} || shini_retval="$1"
# return 0
# fi
shini_retval="$(printf '%s' "$1" | sed -E "s/$2/\1/")" # If you have isses on older systems,
# it may be the non-newer POSIX compliant sed.
# -E should be enabling extended regex mode portably.
}
I get these results - so still a speed improvement compared to before but there is still room for improvement:
tc@piCorePlayer:~/www/cgi-bin$ time ash ./test_perf.sh > /dev/null
Command exited with non-zero status 1
real 1m 43.32s
user 0m 15.56s
sys 0m 32.17s
So this section needs further changes. I think it is this
shini_retval=${BASH_REMATCH[1]}
that upset busybox.
If I test this section I get this error:
tc@piCorePlayer:~/www/cgi-bin$ expr 'abc123' : '^abc[0-9]*'
expr: warning: '^abc[0-9]*': using '^' as the first character
of a basic regular expression is not portable; it is ignored
6
I managed to get the time down to "real 1m 27.95s" if I changed the script to this (t I think I then force it to use your new method?
shini_regex_match()
{
# $KSH_VERSION (I'm told) only exists on ksh 93 and above, which supports regex matching.
if [ -n "$BASH_VERSINFO" ] && [ "$BASH_VERSINFO" -ge 3 ] || \
[ -n "$ZSH_VERSION" ] || \
[ -n "$KSH_VERSION" ]; then
[[ "$1" =~ $2 ]] && return 0 || return 1
fi
# ash/dash, and possibly others
# if [ "$EXPR_REGEX" -eq 0 ]; then
expr "$1" : "$2" > /dev/null 2>&1
return $?
# fi
printf '%s' "$1" | grep -qe "$2"
return $?
}
shini_regex_replace()
{
# if [ -n "$BASH_VERSINFO" ] && [ "${BASH_VERSINFO}" -ge 3 ] || \
# [ -n "$ZSH_VERSION" ]; then
# [[ "$1" =~ $2 ]] && shini_retval=${BASH_REMATCH[1]} || shini_retval="$1"
# return 0
#fi
shini_retval="$1"
# shini_retval="$(printf '%s' "$1" | sed -E "s/$2/\1/")" # If you have isses on older systems,
# it may be the non-newer POSIX compliant sed.
# -E should be enabling extended regex mode portably.
}
tc@piCorePlayer:~/www/cgi-bin$ time ash ./test_perf.sh > /dev/null
Command exited with non-zero status 1
real 1m 27.95s
user 0m 15.64s
sys 0m 33.83s
Regarding this issue:
shini_retval=${BASH_REMATCH[1]}
You are correct. It appears dash
behaves differently, allowing the [1]
syntax. busybox's ash
refuses to process the script at all with it there. I'll have to figure a way around that.
I think when you "force" it to use the expr
mode, all you've done (at least, testing on this busybox VM image I found!) is remove the if
condition. So you're seeing a ~15s increase in performance just because there's no if
statement.
I'm guessing the rasberrypi hardware (is that what you're running the test on?) is just woefully less powerful than the busybox VM I have (which completes in 3.8s).
The test_perf.sh
script is running against a fairly large INI file (1917 lines), what kind of performance do you get if you replace /tests/php.ini
with a smaller example (like the one you'll be using for your project)?
Assuming the if
condition is running that slowly, you'll get further performance gains by completely stripping down that first function:
shini_regex_match()
{
expr "$1" : "$2" > /dev/null 2>&1
return $?
}
Thanks for working on this - it is good you found a busybox environtment. I noticed that if I added a debug notice like this:
shini_regex_replace()
{
# if [ -n "$BASH_VERSINFO" ] && [ "${BASH_VERSINFO}" -ge 3 ] || \
# [ -n "$ZSH_VERSION" ]; then
# [[ "$1" =~ $2 ]] && shini_retval=${BASH_REMATCH[1]} || shini_retval="$1"
# return 0
#fi
echo "we are here 3"
shini_retval="$(printf '%s' "$1" | sed -E "s/$2/\1/")" # If you have isses on older systems,
# it may be the non-newer POSIX compliant sed.
# -E should be enabling extended regex mode portably.
}
The the terminal shows that this function get called two or three times for each variable?
tc@piCorePlayer:~/www/cgi-bin$ time ash ./test_perf.sh
..
..
..
[opcache] opcache.validate_permission = 0
we are here 3
we are here 3
[opcache] opcache.validate_root = 0
we are here 3
we are here 3
we are here 3
[curl] curl.cainfo =
we are here 3
we are here 3
we are here 3
[openssl] openssl.cafile =
we are here 3
we are here 3
[openssl] openssl.capath =
Command exited with non-zero status 1
Hi Did you find time to look at this issue:
Regarding this issue:
shini_retval=${BASH_REMATCH[1]}
You are correct. It appears dash behaves differently, allowing the [1] syntax. busybox's ash refuses to process the script at all with it there. I'll have to figure a way around that.