Name space enabled NVMe devices using new /dev/disk/by-id links

This document (000021131) is provided subject to the disclaimer at the end of this document.

Environment

SUSE Linux Enterprise Server 15 SP 5
SUSE Linux Enterprise Server 15 SP 4

Situation

PCIe SSD NVMe devices with namespace support were added to the server. Multiple namespaces were added to the NVMe devices.

Server host configured as a NVMe TCP initiator. It discovers and connects to multiple NVMe TCP namespaces.

The following /dev/disk/by-id links were observed:

# ls -l /dev/disk/by-id
lrwxrwxrwx 1 root root  13 Jun 26 04:40 nvme-Dell_NVMe_PM1743_RI_E3.S_1.92TB_S778NG0TA00023 -> ../../nvme3n4
lrwxrwxrwx 1 root root  13 Jun 26 04:40 nvme-Dell_NVMe_PM1743_RI_E3.S_1.92TB_S778NG0TA00087 -> ../../nvme1n4
lrwxrwxrwx 1 root root  13 Jun 26 04:40 nvme-Dell_NVMe_PM1743_RI_E3.S_1.92TB_S778NG0TA00101 -> ../../nvme2n4

After the I/O is run on the namespace, the link target changed and points to a different namespace. The issue occurs with both NVMe TCP and local/PCIE SSD namespaces.

NOTE: Other conditions (such as a system reboot) may cause the link to change. As such, the use of this link is never recommended on NVMe devices with namespaces enabled.

# ls -l /dev/disk/by-id                                                                    Changed - vvvvvvv
lrwxrwxrwx 1 root root  13 Jun 26 04:55 nvme-Dell_NVMe_PM1743_RI_E3.S_1.92TB_S778NG0TA00023 -> ../../nvme3n2
lrwxrwxrwx 1 root root  13 Jun 26 04:55 nvme-Dell_NVMe_PM1743_RI_E3.S_1.92TB_S778NG0TA00087 -> ../../nvme1n3
lrwxrwxrwx 1 root root  13 Jun 26 04:55 nvme-Dell_NVMe_PM1743_RI_E3.S_1.92TB_S778NG0TA00101 -> ../../nvme2n2

Other links such as the following remained consistent.

# ls -l /dev/disk/by-id
lrwxrwxrwx 1 root root  13 Jun 26 04:40 nvme-eui.3737383054a00023002538470000012c -> ../../nvme3n4

A single namespace NVMe does not change the target link.

Resolution

The issue is resolved in a systemd maintenance update, version systemd-249.16-150400.8.28.3 or higher.

Use the new style /dev/disk/by-id links that include a trailing underscore and the namespace ID. For example,

# ls -l /dev/disk/by-id
lrwxrwxrwx 1 root root   13 Jun 26 05:39 nvme-Dell_NVMe_PM1743_RI_E3.S_1.92TB_S778NG0TA00023_1 -> ../../nvme3n1
lrwxrwxrwx 1 root root   13 Jun 26 05:39 nvme-Dell_NVMe_PM1743_RI_E3.S_1.92TB_S778NG0TA00023_2 -> ../../nvme3n2
lrwxrwxrwx 1 root root   13 Jun 26 05:39 nvme-Dell_NVMe_PM1743_RI_E3.S_1.92TB_S778NG0TA00023_3 -> ../../nvme3n3
lrwxrwxrwx 1 root root   13 Jun 26 05:39 nvme-Dell_NVMe_PM1743_RI_E3.S_1.92TB_S778NG0TA00023_4 -> ../../nvme3n4

The old style link does not include the namespace and should not be used.

In an NVMe TCP scenario, use the links with the namespace ID. For example,

# ls -l /dev/disk/by-id
lrwxrwxrwx. 1 root root 13 Feb  2 16:11 /dev/disk/by-id/nvme-dellemc-powerstore_72SZ9D3_164 -> ../../nvme2n1
lrwxrwxrwx. 1 root root 13 Feb  2 16:11 /dev/disk/by-id/nvme-dellemc-powerstore_72SZ9D3_165 -> ../../nvme2n2
lrwxrwxrwx. 1 root root 13 Feb  2 16:11 /dev/disk/by-id/nvme-dellemc-powerstore_72SZ9D3_166 -> ../../nvme2n3

Do not use the links without the namespace ID as in:

# ls -l /dev/disk/by-id                                                               
lrwxrwxrwx. 1 root root 13 Feb  2 16:11 /dev/disk/by-id/nvme-dellemc-powerstore_72SZ9D3 -> ../../nvme2n3

Cause

The old style /dev/disk/by-id link names were retained for backward compatibility, but should not be used.

Status

Reported to Engineering

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.