Recover from LVM mismatched Metadata and missing PV
This document (000021791) is provided subject to the disclaimer at the end of this document.
Environment
SUSE Linux Enterprise Server 15 SP6
Situation
The Server has a Volume Group with Two Physical Volumes:
srv1:~ # pvs
PV VG Fmt Attr PSize PFree
/dev/sdb mainvg lvm2 a-- 2.00g 0
/dev/sdc mainvg lvm2 a-- 1020.00m 92.00m
After a while, one PV got missing:
srv1:~ # pvs
WARNING: Couldn't find device with uuid RhZ9ee-0M5r-MZ8D-RS5j-7PTG-BHzi-WsvjQQ.
WARNING: VG mainvg is missing PV RhZ9ee-0M5r-MZ8D-RS5j-7PTG-BHzi-WsvjQQ (last written to /dev/sdb).
WARNING: Couldn't find all devices for LV mainvg/lv01 while checking used and assumed devices.
PV VG Fmt Attr PSize PFree
/dev/sdc mainvg lvm2 a-- 1020.00m 92.00m
[unknown] mainvg lvm2 a-m 2.00g 0
In the meanwhile, with above situation, the VG was modified. E.g., adding a tag:
srv1:~ # vgchange --addtag mySupperTag mainvg
WARNING: Couldn't find device with uuid RhZ9ee-0M5r-MZ8D-RS5j-7PTG-BHzi-WsvjQQ.
WARNING: VG mainvg is missing PV RhZ9ee-0M5r-MZ8D-RS5j-7PTG-BHzi-WsvjQQ (last written to /dev/sdb).
WARNING: Couldn't find device with uuid RhZ9ee-0M5r-MZ8D-RS5j-7PTG-BHzi-WsvjQQ.
WARNING: Couldn't find device with uuid RhZ9ee-0M5r-MZ8D-RS5j-7PTG-BHzi-WsvjQQ.
Volume group "mainvg" successfully changed
After adding the missing original PV disk back, we have two warnings:
- LVM Metadata mismatch
- Missing PV
srv1:~ # pvscan --cache
pvscan[3472] PV /dev/sdb online.
WARNING: VG mainvg was previously updated while PV /dev/sdb was missing.
pvscan[3472] PV /dev/sdc online.
srv1:~ # pvs
WARNING: ignoring metadata seqno 11 on /dev/sdb for seqno 12 on /dev/sdc for VG mainvg.
WARNING: Inconsistent metadata found for VG mainvg.
See vgck --updatemetadata to correct inconsistency.
WARNING: VG mainvg was previously updated while PV /dev/sdb was missing.
WARNING: VG mainvg was missing PV /dev/sdb RhZ9ee-0M5r-MZ8D-RS5j-7PTG-BHzi-WsvjQQ.
PV VG Fmt Attr PSize PFree
/dev/sdb mainvg lvm2 a-m 2.00g 0
/dev/sdc mainvg lvm2 a-- 1020.00m 92.00m
Resolution
To fix LVM Metadata mismatch run: vgck --updatemetadata <VG>
srv1:~ # vgck --updatemetadata mainvg
WARNING: ignoring metadata seqno 11 on /dev/sdb for seqno 12 on /dev/sdc for VG mainvg.
WARNING: Inconsistent metadata found for VG mainvg.
See vgck --updatemetadata to correct inconsistency.
WARNING: VG mainvg was previously updated while PV /dev/sdb was missing.
WARNING: VG mainvg was missing PV /dev/sdb RhZ9ee-0M5r-MZ8D-RS5j-7PTG-BHzi-WsvjQQ.
WARNING: VG mainvg was previously updated while PV /dev/sdb was missing.
WARNING: updating old metadata to 13 on /dev/sdb for VG mainvg.
WARNING: VG mainvg was previously updated while PV /dev/sdb was missing.
To recover from Missing PV run: vgextend <VG> <MISSING_PV> --restoremissing
srv1:~ # vgextend mainvg /dev/sdb --restoremissing
WARNING: VG mainvg was previously updated while PV /dev/sdb was missing.
WARNING: VG mainvg was missing PV /dev/sdb RhZ9ee-0M5r-MZ8D-RS5j-7PTG-BHzi-WsvjQQ.
WARNING: VG mainvg was previously updated while PV /dev/sdb was missing.
Volume group "mainvg" successfully extended
srv1:~ # vgchange -a y mainvg
1 logical volume(s) in volume group "mainvg" now active
srv1:~ # vgs
VG #PV #LV #SN Attr VSize VFree
mainvg 2 1 0 wz--n- 2.99g 92.00m
Cause
The LVM metadata became inconsistent as a result of updating the Volume Group (VG) while a Physical Volume (PV) belonging to it was missing.
Additional Information
Check the LVM Metadata from PV:
srv1:~ # pvck --dump metadata /dev/sdb | egrep "seqno =|tags ="
seqno = 11
srv1:~ # pvck --dump metadata /dev/sdc | egrep "seqno =|tags ="
seqno = 12
tags = ["mySupperTag"]
After running: vgck --updatemetadata mainvg
srv1:~ # pvck --dump metadata /dev/sdb | egrep "seqno =|tags ="
seqno = 13
tags = ["mySupperTag"]
srv1:~ # pvck --dump metadata /dev/sdc | egrep "seqno =|tags ="
seqno = 13
tags = ["mySupperTag"]
Disclaimer
This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.
- Document ID:000021791
- Creation Date: 15-Apr-2025
- Modified Date:25-Apr-2025
-
- SUSE Linux Enterprise Server
For questions or concerns with the SUSE Knowledgebase please contact: tidfeedback[at]suse.com