Disksort vs Max_Pending
Tyson vs Holyfield
Good news… here you can get a little script to enable, disable, and check the disksort flag on your disks (not persistent after reboot, take a look on my other post to a link for how to do it).
Take a look at the begining of the script so you will see if you need to change it or not. It’s important to see what is enabled or disabled on your disks, because i’m not preserving old settings that are part of that “Byte“.
Disksort script:
#!/usr/bin/bash
#byLeal (https://www.eall.com.br)
######################################################
# DO NOT TRY IT IN PRODUCTION. USE AT YOUR OWN RISK! #
######################################################
# 2010/09/09
# Simple Script to check and/or enable/disable disksort in ALL disks
# on the system...
# IMPORTANT: This script will change the Byte with the bit for
# disksort_disable to 0xc. Leaving the configuration like this:
#
# unsigned un_f_pm_is_enabled :1 = 0
# unsigned un_f_watcht_stopped :1 = 0
# unsigned un_f_pkstats_enabled :1 = 0x1
# unsigned un_f_disksort_disabled :1 = 0x1
# unsigned un_f_lun_reset_enabled :1 = 0
# unsigned un_f_doorlock_supported :1 = 0
# unsigned un_f_start_stop_supported :1 = 0
# unsigned un_f_reserved1 :1 = 0
#
# Ex.: If you want to preserve the "power management" enable bit,
# you will need to change the "v0t12" for "v0t13".
#
NRDISKS=`iostat -x | grep ^sd | wc -l`;
x=1;
if [ $# != 1 ];
then
echo "disksort options: \"enable\", \"disable\", or \"check\""
exit 1;
fi
if [ $1 != "check" ] && [ $1 != "disable" ] && [ $1 != "enable" ];
then
echo "disksort: Unknown option \"$1\"..."
exit 1;
fi
if [ $1 == disable ]; then
mdbflag="v0t12";
fi
if [ $1 == enable ]; then
mdbflag="v0t4";
fi
if [ $1 == check ]; then
while [ $x -le $NRDISKS ];
do echo -n "sd$x: ";
echo "*sd_state::softstate 0t$x | ::print -at 'struct sd_lun' \
un_f_disksort_disabled" | mdb -k;
let x=x+1;
done
fi
if [ $1 == disable ] || [ $1 == enable ]; then
while [ $x -le $NRDISKS ];
do echo -n "sd$x: ";
for y in `echo "*sd_state::softstate 0t$x | \
::print -at 'struct sd_lun' un_f_pm_is_enabled" | \
mdb -k | awk '{print $1}'`;
do
echo "$y/$mdbflag" | mdb -kw;
done;
let x=x+1;
done
fi
exit 0;
Bad news… i did not get a huge difference disabling the disksort regarding to latency times. Very strange… as the parameter is “live” on kernel space, it should make a big win. So, i did shrink the waitq queue and here you can see the latency times before (./scsi.d-1.18 -D QUIET -D PERF_REPORT -D REPORT_TARGET -D REPORT_LUN -n tick-1m):
value ------------- Distribution ------------- count 32768 | 0 65536 | 459 131072 |@@@@@@@@@ 31294 262144 |@@ 6170 524288 |@@@ 11513 1048576 |@@ 5928 2097152 |@ 2780 4194304 |@ 4244 8388608 |@@@@ 12426 16777216 |@@ 8295 33554432 |@@@@ 12533 67108864 |@@@@@@@@@@@ 37983 134217728 |@ 3018 268435456 | 581 536870912 | 60 1073741824 | 42 2147483648 | 1 4294967296 | 0
and after shrink zfs_vdev_max_pending…
value ------------- Distribution ------------- count 32768 | 0 65536 | 1274 131072 |@@@@@@@ 32486 262144 |@@ 9834 524288 |@@ 6724 1048576 |@ 3749 2097152 |@ 4350 4194304 |@@ 6638 8388608 |@@@@@@ 27361 16777216 |@@@@@@@@@@@@@@@@@ 74990 33554432 |@@ 6982 67108864 | 1369 134217728 | 744 268435456 | 229 536870912 | 46 1073741824 | 34 2147483648 | 0
and the winner is… ;-)
peace
