SES4: Deleting files within a Rados Block Device does not free up space on the cluster.

This document (7022066) is provided subject to the disclaimer at the end of this document.

Environment

SUSE Enterprise Storage 4

Situation

Customer is using Rados Block Devices to store data. Customer moved data from one rbd device to another, expecting the first rbd device to release the freed space back to the SES cluster.

Resolution

There is good news and a few options to address this issue. 
  • If the goal is to free up space, so osd's are not marked as full, then it may be best to add more disks to the cluster.
  • If space is available, create a new rbd device and copy the data to the new device, then delete the original rbd device. Deleting the rbd device will return the space to SES.
  • In some cases "fstrim" can return the "freed" space back to SES, assuming that the stack, filesystem, drivers, kernel, HBA, and hard drives support the functionality. fstrim should be run on the filesystem within the rbd block device.   
Example:
List block devices:
rbd list PooName
Map the image to a new block device.
rbd map --pool PoolName ImageName
List all mapped devices:
rbd showmapped
Mount the device:
mount /dev/rbd0 /mnt
run fstrim:
fstrim /mnt
  • Mounting the file system with "-o discard" could provide same results with out having to run "fstrim".
Example: mount -o discard /dev/rbd0 /mnt

Cause

It is normal behavior for a rbd device to not return "freed" space back to the SES cluster when a files are deleted.  Only the pointers to the files are removed. The blocks of data that made up the files, remain on the file system. The filesystem within the rbd device will recognize that there is free space, however SES is not aware that the files were deleted and can not distinguish deleted files from files that have not been deleted.  This behavior is the same, if the operation were done on a normal hard drive with a posix file system.

Additional Information

https://bugzilla.suse.com/show_bug.cgi?id=1062152

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.

  • Document ID:7022066
  • Creation Date: 11-Oct-2017
  • Modified Date:03-Mar-2020
    • SUSE Enterprise Storage

< Back to Support Search

For questions or concerns with the SUSE Knowledgebase please contact: tidfeedback@suse.com

SUSE Support Forums

Get your questions answered by experienced Sys Ops or interact with other SUSE community experts.

Join Our Community

Support Resources

Learn how to get the most from the technical support you receive with your SUSE Subscription, Premium Support, Academic Program, or Partner Program.


SUSE Customer Support Quick Reference Guide SUSE Technical Support Handbook Update Advisories
Support FAQ

Open an Incident

Open an incident with SUSE Technical Support, manage your subscriptions, download patches, or manage user access.

Go to Customer Center