Name space enabled NVMe devices using new /dev/disk/by-id links
This document (000021131) is provided subject to the disclaimer at the end of this document.
Environment
SUSE Linux Enterprise Server 15 SP 4
Situation
Server host configured as a NVMe TCP initiator. It discovers and connects to multiple NVMe TCP namespaces.
The following /dev/disk/by-id links were observed:
# ls -l /dev/disk/by-id lrwxrwxrwx 1 root root 13 Jun 26 04:40 nvme-Dell_NVMe_PM1743_RI_E3.S_1.92TB_S778NG0TA00023 -> ../../nvme3n4 lrwxrwxrwx 1 root root 13 Jun 26 04:40 nvme-Dell_NVMe_PM1743_RI_E3.S_1.92TB_S778NG0TA00087 -> ../../nvme1n4 lrwxrwxrwx 1 root root 13 Jun 26 04:40 nvme-Dell_NVMe_PM1743_RI_E3.S_1.92TB_S778NG0TA00101 -> ../../nvme2n4After the I/O is run on the namespace, the link target changed and points to a different namespace. The issue occurs with both NVMe TCP and local/PCIE SSD namespaces.
NOTE: Other conditions (such as a system reboot) may cause the link to change. As such, the use of this link is never recommended on NVMe devices with namespaces enabled.
# ls -l /dev/disk/by-id Changed - vvvvvvv lrwxrwxrwx 1 root root 13 Jun 26 04:55 nvme-Dell_NVMe_PM1743_RI_E3.S_1.92TB_S778NG0TA00023 -> ../../nvme3n2 lrwxrwxrwx 1 root root 13 Jun 26 04:55 nvme-Dell_NVMe_PM1743_RI_E3.S_1.92TB_S778NG0TA00087 -> ../../nvme1n3 lrwxrwxrwx 1 root root 13 Jun 26 04:55 nvme-Dell_NVMe_PM1743_RI_E3.S_1.92TB_S778NG0TA00101 -> ../../nvme2n2Other links such as the following remained consistent.
# ls -l /dev/disk/by-id lrwxrwxrwx 1 root root 13 Jun 26 04:40 nvme-eui.3737383054a00023002538470000012c -> ../../nvme3n4A single namespace NVMe does not change the target link.
Resolution
Use the new style /dev/disk/by-id links that include a trailing underscore and the namespace ID. For example,
# ls -l /dev/disk/by-id lrwxrwxrwx 1 root root 13 Jun 26 05:39 nvme-Dell_NVMe_PM1743_RI_E3.S_1.92TB_S778NG0TA00023_1 -> ../../nvme3n1 lrwxrwxrwx 1 root root 13 Jun 26 05:39 nvme-Dell_NVMe_PM1743_RI_E3.S_1.92TB_S778NG0TA00023_2 -> ../../nvme3n2 lrwxrwxrwx 1 root root 13 Jun 26 05:39 nvme-Dell_NVMe_PM1743_RI_E3.S_1.92TB_S778NG0TA00023_3 -> ../../nvme3n3 lrwxrwxrwx 1 root root 13 Jun 26 05:39 nvme-Dell_NVMe_PM1743_RI_E3.S_1.92TB_S778NG0TA00023_4 -> ../../nvme3n4The old style link does not include the namespace and should not be used.
In an NVMe TCP scenario, use the links with the namespace ID. For example,
# ls -l /dev/disk/by-id lrwxrwxrwx. 1 root root 13 Feb 2 16:11 /dev/disk/by-id/nvme-dellemc-powerstore_72SZ9D3_164 -> ../../nvme2n1 lrwxrwxrwx. 1 root root 13 Feb 2 16:11 /dev/disk/by-id/nvme-dellemc-powerstore_72SZ9D3_165 -> ../../nvme2n2 lrwxrwxrwx. 1 root root 13 Feb 2 16:11 /dev/disk/by-id/nvme-dellemc-powerstore_72SZ9D3_166 -> ../../nvme2n3Do not use the links without the namespace ID as in:
# ls -l /dev/disk/by-id lrwxrwxrwx. 1 root root 13 Feb 2 16:11 /dev/disk/by-id/nvme-dellemc-powerstore_72SZ9D3 -> ../../nvme2n3
Cause
Status
Disclaimer
This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.
- Document ID:000021131
- Creation Date: 07-Jul-2023
- Modified Date:16-Aug-2023
-
- SUSE Linux Enterprise Server
For questions or concerns with the SUSE Knowledgebase please contact: tidfeedback[at]suse.com