node.session.timeo.replacement_timeout/recovery_tmo always changes from 120 to 5 after a while
This document (000021146) is provided subject to the disclaimer at the end of this document.
Environment
SUSE linux Enterprise Server all Service Packs
Situation
iSCSI option node.session.timeo.replacement_timeout is set to default value 120:
yet, during target outage, kernel reports I/O errors to offline device after 5 seconds:
n1:~ # iscsiadm -m node --targetname iqn.shared_storage:12sp5 -o show | grep node.session.timeo.replacement_timeout node.session.timeo.replacement_timeout = 120 node.session.timeo.replacement_timeout = 120 node.session.timeo.replacement_timeout = 120
yet, during target outage, kernel reports I/O errors to offline device after 5 seconds:
Jul 21 12:42:36 n1 kernel: session2: session recovery timed out after 5 secs Jul 21 12:42:37 n1 kernel: session1: session recovery timed out after 5 secs Jul 21 12:42:37 n1 kernel: sd 7:0:0:0: rejecting I/O to offline device Jul 21 12:42:37 n1 kernel: sd 7:0:0:0: [sdb] killing request Jul 21 12:42:37 n1 kernel: sd 7:0:0:0: [sdb] FAILED Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK Jul 21 12:42:37 n1 kernel: sd 7:0:0:0: [sdb] CDB: Read(10) 28 00 00 00 00 00 00 00 01 00 Jul 21 12:42:37 n1 kernel: print_req_error: I/O error, dev sdb, sector 0 Jul 21 12:42:37 n1 kernel: sd 7:0:0:0: rejecting I/O to offline device
Resolution
If /etc/multipath.conf doesn't exist, create it and add following content:
devices {
device {
vendor IET
product VIRTUAL-DISK
fast_io_fail_tmo 120
}
replace vendor and product to appropriate values for your storage.
The values can be obtained from lsscsi command:
n1:~ # lsscsi [7:0:0:0] disk IET VIRTUAL-DISK 0 /dev/sda [7:0:0:1] disk IET VIRTUAL-DISK 0 /dev/sdb [7:0:0:2] disk IET VIRTUAL-DISK 0 /dev/sdd [8:0:0:0] disk IET VIRTUAL-DISK 0 /dev/sdc
Apply the new settings with multipath command, for example: multipath -d
Cause
multipath command resets the default iSCSI timeout from 120 to 5:
n1:~ # cat /sys/class/iscsi_session/session*/recovery_tmo 120 120 n1:~ # multipath -v 3 -d n1:~ # cat /sys/class/iscsi_session/session*/recovery_tmo 5 5Even though multipath is not used, the command multipath -v3 -d is executed by supportconfig, which support usually requires for analysis.
Disclaimer
This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.
- Document ID:000021146
- Creation Date: 21-Jul-2023
- Modified Date:24-Jul-2023
-
- SUSE Linux Enterprise Server
For questions or concerns with the SUSE Knowledgebase please contact: tidfeedback[at]suse.com
