Time For Switching – SLES 12 SP3 In The Public Cloud

It was release day September 7, 2017, and SLES 12 SP3 and SLES 12 SP3 For SAP Application images are available in Amazon EC2, Google Compute Engine, and Microsoft Azure. Give them a whirl. If you plan on attending SUSECon in Prague in a couple of weeks you can meet some members of the Public Cloud team that make all this magic happen. And if you have no plans yet, well there’s still time to register, come out and see us.

What’s this mean to you? Well, when starting anything new you should start using the new images, on-demand or BYOS as the clock on the SLES 12 SP2 images has started ticking. For running instances it is time to start thinking about migration, and there are some caveats that I will get to in a couple of minutes. But first things first, how to find the new images.

Each framework has a different mechanism to get the new images accessible via the web console and thus making them available via point and click. We have little influence over the speed of this process and you will need to keep an eye out for the images in the web console. Of course all frameworks offer a search feature in their web console and searching for “12-sp3” or “v20170907” will find the images if they are not yet in the place where you expect them. For those that use the command line tools to run their instances the latest and greatest images can be found via pint, as always. For new goodies in SLES 12 SP3 please consult the Release Notes. As far as Public Cloud related new goodies, well those are not tied to a release. The Public Cloud module has a “continuous integration” life cycle and we get to update things such as command line tools along the way and do not have to wait for new SP releases. Therefore, there’s nothing really to announce on that front. A couple of highlights are on the networking front for Amazon EC2 and Azure. Relevant to Amazon EC2 is that the latest ENA (Elastic Network Adapter) driver is integrated in the SLES 12 SP3 kernel (it is also available via update in the SP2 kernel) and for Microsoft Azure SR-IOV bonding is also fully integrated into the kernel and happens auto-magically.

Now on to the migration part as I have made those looking for migration caveats wait long enough. First lets cover SLES migration, and I will focus only on the SLES 12 SP2 to SLES 12 SP3 migration as other migration paths have been described previously. SLES For SAP Applications will be covered after the SLES section. One improvement is that we are getting away from framework dependent “if-then-else” parts of the migration, this should be the last time we deal with this. On the other hand we took a step back in ease of migration by picking up a bug during the SLES 12 SP2 cycle. This bug introduces another “if-then-else” flow that I will describe below. During the SLES 12 SP2 cycle SUSE announced the availability of the HPC module which is distinctly different from the special HPC images offered in Microsoft Azure, and I will get to those later.

In an effort to make the packages for HPC easily available the product package for HPC was included in the image builds for SLES 12 SP2 and SLES 12 SP2 For SAP at some point such that the HPC repository would auto-magically get registered for on-demand instances. Following the theme that “no good deed shall go unpunished”, if we look at this from a more cynical point of view, we also ended up picking up a bug that makes migration a bit more cumbersome, this time. With that as introduction lets get to the action.

Before migrating, and as a note what follows is targeted for on-demand instance users, you want to make sure your instance already has the latest and greatest SLES 12 SP2 updates. All commands need root permission. Start with

zypper up

If you get a kernel update during this update you do not really need to reboot your instance. Next lets get the migration plugin, if you do not already have it

zypper in zypper-migration-plugin

Now to the HPC module complication, run

rpm -qa | grep hpc

If nothing shows up, meaning you started from an image that does not have the HPC module registered then you do not have the problem, easy enough, and you get to skip over the next few steps. If you do have the packages we need to double check the proclaimed module version, thus run

zypper products | grep -i hpc

If in this output you see a version string of “12.2-0” then there would be a migration issue that we want to nip in the butt before we run the migration process. If the version string is “12-0” you’re all set. For those with the buggy version the solution is relatively simple, as documented in the Release Notes. But before we go down that path, since I have no idea from which image your instance originated lets double check that you have the file /etc/SUSEConnect

ls /etc/SUSEConnect

If the file does exists we are good to go, if not lets create it. This is framework dependent.

For AWS the content should be:

---
insecure: false
url: https://smt-ec2.susecloud.net

For GCE the content should be:

---
insecure: false
url: https://smt-gce.susecloud.net

For Azure the content should be:

---
insecure: false
url: https://smt-azure.susecloud.net

Easy enough. Now back to managing the HPC version mismatch problem. If you do have “12.2-0” in the output from the “zypper products | grep -i hpc” command then you want to run:

rpm -e sle-module-hpc-release-POOL sle-module-hpc-release
SUSEConnect -p sle-module-hpc/12/x86_64

If you do not have the HPC module on your system and you want access to it you can also run “SUSEConnect -p sle-module-hpc/12/x86_64”

Now after all of this preparation we are ready for the migration, yippee

zypper migration

After the process is complete reboot your instance and you are all set. Enjoy SLES 12 SP3

Oh, and to avoid any confusion on login you might want to change your motd file

sed -i s/SP2/SP3/ /etc/motd

Now on to SLES For SAP. Given that SLES 12 SP3 For SAP Applications is brand new not all SAP products have been certified yet. However there is a large number of SAP applications for which the certification on SLES 12 carries over to SLES 12 SP3. Also if you are running instances based on SLES 12 SP1 you can follow the exact same steps as the same caveats apply for SLES 12 SP1 For SAP and SLES 12 SP2 For SAP. As with the SLES migration this applies to on-demand instances that are registered to the SUSE operated infrastructure in your cloud framework of choice.

Update 2018-07-28

Well the “exact same steps” in combination with the commands that followed was certainly misleading, sorry. I will leave the original section as a reference below but you should follow this new and updated section, and yes, this time I mean it, use the exact same steps for SLES 12 SP1 For SAP and SLES 12 SP2 For SAP migrations.

First we are going to figure out what needs to be removed:

rpm -qa | grep release | egrep 'sles|hpc|ha' | grep -v notes

Any package returned as the output needs to be removed. Depending on the origin of the instance this may return a list such as this:

sle-module-hpc-release-POOL-12.2-1.1.x86_64
sle-ha-release-POOL-12.2-1.125.x86_64
sle-module-toolchain-release-POOL-12-8.1.x86_64
sle-module-toolchain-release-12-8.1.x86_64
sle-ha-release-12.2-1.125.x86_64
sle-module-hpc-release-12.2-1.1.x86_64

or similar to this:

sles-release-POOL-12.1-1.331.x86_64
sles-release-12.1-1.331.x86_64
sle-ha-release-POOL-12.1-1.64.x86_64
sle-ha-release-12.1-1.64.x86_64

And of course we did fix the issue in the image at some point and you may get nothing back.

For those that get nothing back from this command the “standard” zypper up, zypper migration process applies. For those that get something back the  next step is to remove the packages in question, for example this may look like this:

zypper rm sle-module-hpc-release-POOL-12.2-1.1.x86_64 \

sle-ha-release-POOL-12.2-1.125.x86_64 \

sle-module-toolchain-release-POOL-12-8.1.x86_64 \

sle-module-toolchain-release-12-8.1.x86_64 \

sle-ha-release-12.2-1.125.x86_64 \

sle-module-hpc-release-12.2-1.1.x86_64

Do not cut and paste the above command it will not necessarily match what is on your system, you need to customize the “zypper rm” command based on the package list obtained form the “rpm -qa” command shown previously

The next step form the original instructions still applies for those running on AWS:

rm /etc/zypp/repos.d/aws-ena*

This may return a message that the file does not exists, which is OK. If the file does not exist it means the ENA driver is already built in and we didn’t need to pull it from the SUSE Solid Driver build service.

Now update you system:

zypper up

Force a new registration to clean up the repositories known to the system:

registercloudguest --force-new

This step is necessary to sort out the repositories on your running instance, ignore the certificate warning. And finally we arrive where we want to be, an instance that can be migrated

zypper migration

If you want the HPC module back run

SUSEConnect -p sle-module-hpc/12-0/x86_64

This concludes the updated part of the blog. The original text, only for reference purposes is blow and marked accordingly.

Original section for reference only, do not use. Use the updated section above

We are going to start not by updating the system but by getting rid of stuff that interferes with the migration process:

zypper rm sles-release-POOL-12.1-1.331.x86_64 \
sles-release-12.1-1.331.x86_64 \
sle-ha-release-POOL-12.1-1.64.x86_64 \
sle-ha-release-12.1-1.64.x86_64

While this will show some packages to be removed that should ordinarily give you reason to pause, it is OK to proceed. A total of 7 packages should be removed.

Next you get to do something that under ordinary circumstances only causes trouble, so for this one time and one time only use SUSEConnect on an on-demand instance.

SUSEConnect -d

This will produce an error

“Error: SCC returned ‘Internal Server Error: Please contact your Administrator’ (500)”

Which is OK, you are not registered to SCC. This is an artifact of the update infrastructure. Locally the changes we want to happen still happened. In AWS EC2 we want to get rid of a special repo that exists only on SLES 12 SP1 For SAP instances to provide ENA access and will no longer be needed as in SLES 12 SP2 & SP3 For SAP, as in SLES 12 SP2 & SP3 the ENA driver is part of the kernel shipped by SUSE.

rm /etc/zypp/repos.d/aws-ena*

Now let zypper clean up all of it’s knowledge of the state of the system

zypper up

You’ll see a bunch of “Removing repository …” messages scroll across the screen. With this we have the system in a state where we can start from scratch with a registration:

registercloudguest --force-new

Ignore the warning about the certificate. Once the registration is finished lets make sure we have the latest and greatest:

zypper up

If you run into file conflicts type “yes” to let zypper proceed and update the packages. Now the instance is in a state so you can jump back up and follow the instructions for SLES, which includes the handling of the HPC module. On SLES For SAP instances the HPC module handling also depends on the origin of your migration. If the module is not already registered and you are starting from SLES 12 SP1 For SAP you want to wait with the HPC module registration until after the migration is complete, if you care to have the module repository set up.

End replaced section

And as a last topic for today a few words about HPC images in Azure, a section I could title “Unsuccessfully banging my head against the wall”. Those interested in HPC and using the HPC images will have noticed that, first we never released SLES 12 SP2 HPC images in Azure, second there was no mention of HPC w.r.t. migration. In short, yes, we never managed to get all the ducks in a row and produce a fully functioning HPC image for SLES 12 SP2. For SLES 12 SP3 we wanted to make sure we had an image on release day and worked really hard to get there. But once again the gremlins got us and the image release for HPC is delayed. Once we get there, sometime before Thanksgiving I hope, watch this space for a special blog about the HPC images. That means that there will be SLES 12 SP3 based HPC images in Azure in the not too distant future.

For the next time, well the next major release will be SUSE Linux Enterprise 15 and there will not be a migration path initially, but for SUSE Linux Enterprise 12 SP4 we’ll continue to work on cutting down the “if-then-else” conditions for migration.