Timekeeping issues on virtual machines

This document (7015560) is provided subject to the disclaimer at the end of this document.

Environment

SUSE Linux Enterprise Server 15
SUSE Linux Enterprise Server 12
SUSE Linux Enterprise Server 11 SP4

Situation

Network time synchronization is setup on the virtual machine yet something keeps interfering and changing the clock to wrong values.

Resolution

Make sure the host machine has correct time and is synchronized to the same time source. A virtual machine usually gets it initial hardware clock value from it when booting.
Review your NTP configuration and make sure not to use the local clock
- Modify the /etc/ntp.conf file. Comment out the following entries so the local clock will not be used by placing # signs before them:
  #server 127.127.1.0 # local clock
  #fudge 127.127.1.0 stratum 10
Disable guest to host time synchronization if applicable - please see below

KVM:
https://documentation.suse.com/

XEN:
https://documentation.suse.com/
As of XEN version 4 the independent_wallclock has been deprecated in favor of tsc_mode.
Please see our documentation for the respective version you are using.

Citrix XEN:
https://docs.citrix.com/en-us/xenserver/7-1/vms/advanced.html#time-handling-in-linux-vms

VMware:
On newest ESXi the default is that time synchronization between host and guest is disabled.

However even with disabled checkbox in Vmware management interface there are still situations where the sync will happen. See article link below for more information.

Disabling Time Synchronization (1189)
https://kb.vmware.com/s/article/1189

Timekeeping best practices for Linux guests (1006427)
https://kb.vmware.com/s/article/1006427
(has some NTP configuration recommendations)

"Timekeeping in VMware virtual machines"
https://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/techpaper/Timekeeping-In-VirtualMachines.pdf

Other sources:
https://opensource.com/article/17/6/timekeeping-linux-vms

Cause

By design, virtual environments configure Guest virtual machines to sync time with the virtual Host server.

Additional Information

In one situation, the virtual server would jump ahead 50 seconds, within 5 to 15 seconds after NTP time sync occurred. And, we noticed that the Host server was ahead of the NTP source server by 36 seconds. NTP time sync was then forced on the Host server to the NTP source server, bringing it in line with the NTP source server. NTP time sync was again triggered on the virtual server, setting it back to the NTP source server. Again, within a few seconds the virtual server jumped ahead 14 seconds. This 50 / 14 second off-set correlated with the offset the Host server had with the NTP source server, identifying that the virtual server jumped ahead 14 seconds respective to the time of the Host server, identifying a time sync occurred with the virtual guest server to the Host server.

This being a Citrix XEN server, echo 1 > /proc/sys/xen/independent_wallclock was executed on the virtual server followed by a NTP time sync. After which, the virtual server stayed on time with the NTP source server. Additional information is found in the Citrix XEN documentation to configure Guest-to-Host time sync to not occur after reboot.

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.