SUSE Support

Here When You Need Us

Recover from LVM mismatched Metadata and missing PV

This document (000021791) is provided subject to the disclaimer at the end of this document.

Environment

SUSE Linux Enterprise Server 15 SP6


Situation

The Server has a Volume Group with Two Physical Volumes: 

srv1:~ # pvs
  PV         VG     Fmt  Attr PSize    PFree
  /dev/sdb   mainvg lvm2 a--     2.00g     0
  /dev/sdc   mainvg lvm2 a--  1020.00m 92.00m

After a while, one PV got missing: 

srv1:~ # pvs
  WARNING: Couldn't find device with uuid RhZ9ee-0M5r-MZ8D-RS5j-7PTG-BHzi-WsvjQQ.
  WARNING: VG mainvg is missing PV RhZ9ee-0M5r-MZ8D-RS5j-7PTG-BHzi-WsvjQQ (last written to /dev/sdb).
  WARNING: Couldn't find all devices for LV mainvg/lv01 while checking used and assumed devices.
  PV         VG     Fmt  Attr PSize    PFree
  /dev/sdc   mainvg lvm2 a--  1020.00m 92.00m
  [unknown]  mainvg lvm2 a-m     2.00g     0

In the meanwhile, with above situation, the VG was modified. E.g., adding a tag: 

srv1:~ # vgchange --addtag mySupperTag mainvg
  WARNING: Couldn't find device with uuid RhZ9ee-0M5r-MZ8D-RS5j-7PTG-BHzi-WsvjQQ.
  WARNING: VG mainvg is missing PV RhZ9ee-0M5r-MZ8D-RS5j-7PTG-BHzi-WsvjQQ (last written to /dev/sdb).
  WARNING: Couldn't find device with uuid RhZ9ee-0M5r-MZ8D-RS5j-7PTG-BHzi-WsvjQQ.
  WARNING: Couldn't find device with uuid RhZ9ee-0M5r-MZ8D-RS5j-7PTG-BHzi-WsvjQQ.
  Volume group "mainvg" successfully changed

After adding the missing original PV disk back, we have two warnings:

  • LVM Metadata mismatch
  • Missing PV 
srv1:~ # pvscan --cache
  pvscan[3472] PV /dev/sdb online.
  WARNING: VG mainvg was previously updated while PV /dev/sdb was missing.
  pvscan[3472] PV /dev/sdc online.
srv1:~ # pvs
  WARNING: ignoring metadata seqno 11 on /dev/sdb for seqno 12 on /dev/sdc for VG mainvg.
  WARNING: Inconsistent metadata found for VG mainvg.
  See vgck --updatemetadata to correct inconsistency.
  WARNING: VG mainvg was previously updated while PV /dev/sdb was missing.
  WARNING: VG mainvg was missing PV /dev/sdb RhZ9ee-0M5r-MZ8D-RS5j-7PTG-BHzi-WsvjQQ.
  PV         VG     Fmt  Attr PSize    PFree
  /dev/sdb   mainvg lvm2 a-m     2.00g     0
  /dev/sdc   mainvg lvm2 a--  1020.00m 92.00m

 

Resolution

To fix LVM Metadata mismatch run: vgck --updatemetadata <VG> 

srv1:~ # vgck --updatemetadata mainvg
  WARNING: ignoring metadata seqno 11 on /dev/sdb for seqno 12 on /dev/sdc for VG mainvg.
  WARNING: Inconsistent metadata found for VG mainvg.
  See vgck --updatemetadata to correct inconsistency.
  WARNING: VG mainvg was previously updated while PV /dev/sdb was missing.
  WARNING: VG mainvg was missing PV /dev/sdb RhZ9ee-0M5r-MZ8D-RS5j-7PTG-BHzi-WsvjQQ.
  WARNING: VG mainvg was previously updated while PV /dev/sdb was missing.
  WARNING: updating old metadata to 13 on /dev/sdb for VG mainvg.
  WARNING: VG mainvg was previously updated while PV /dev/sdb was missing.

 

To recover from Missing PV run: vgextend <VG> <MISSING_PV> --restoremissing 

srv1:~ # vgextend mainvg /dev/sdb --restoremissing
  WARNING: VG mainvg was previously updated while PV /dev/sdb was missing.
  WARNING: VG mainvg was missing PV /dev/sdb RhZ9ee-0M5r-MZ8D-RS5j-7PTG-BHzi-WsvjQQ.
  WARNING: VG mainvg was previously updated while PV /dev/sdb was missing.
  Volume group "mainvg" successfully extended

srv1:~ # vgchange -a y mainvg
  1 logical volume(s) in volume group "mainvg" now active

srv1:~ # vgs
  VG     #PV #LV #SN Attr   VSize VFree
  mainvg   2   1   0 wz--n- 2.99g 92.00m

 

Cause

The LVM metadata became inconsistent as a result of updating the Volume Group (VG) while a Physical Volume (PV) belonging to it was missing.

Additional Information

Check the LVM Metadata from PV: 

srv1:~ # pvck --dump metadata /dev/sdb | egrep "seqno =|tags ="
seqno = 11

srv1:~ # pvck --dump metadata /dev/sdc | egrep "seqno =|tags ="
seqno = 12
tags = ["mySupperTag"]

After running: vgck --updatemetadata mainvg

srv1:~ # pvck --dump metadata /dev/sdb | egrep "seqno =|tags ="
seqno = 13
tags = ["mySupperTag"]

srv1:~ # pvck --dump metadata /dev/sdc | egrep "seqno =|tags ="
seqno = 13
tags = ["mySupperTag"]

 

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.

  • Document ID:000021791
  • Creation Date: 15-Apr-2025
  • Modified Date:25-Apr-2025
    • SUSE Linux Enterprise Server

< Back to Support Search

For questions or concerns with the SUSE Knowledgebase please contact: tidfeedback[at]suse.com

tick icon

SUSE Support Forums

Get your questions answered by experienced Sys Ops or interact with other SUSE community experts.

tick icon

Support Resources

Learn how to get the most from the technical support you receive with your SUSE Subscription, Premium Support, Academic Program, or Partner Program.

tick icon

Open an Incident

Open an incident with SUSE Technical Support, manage your subscriptions, download patches, or manage user access.