Moving forward – Appliances in a new light

Let’s get on the way back train again for a minute or two, and before I get started let me include a disclaimer; The following includes semi-technical marketing information as a prelude to discussing technical solutions. I will try and keep the marketing speak to a minimum.

Anyway, back to the way back train. Way back when SUSE was part of Novell, and SUSE Studio was born along with it came the so called SUSE Appliance program. The basic idea of the appliance program is pretty simple; as an ISV or systems integrator you sign a contract with SUSE to distribute SUSE Linux Enterprise along with one or more applications in a ready to integrate and ready to run solution package. The solution package may come in a number of form factors, virtual images, integrated with hardware, or self installing appliances on optical media or a USB stick. When the whole thing started the public cloud was already in existence but not many people did really pay attention to it.

Fast forward 5 or 6 six years and the world is a very different place. The appliance program is now called SUSE Embedded and foremost, the Public Cloud, i.e. Amazon Web Service, Google Compute Engine, and Microsoft Azure are the talk of the town. As we all know, when it comes to software and anything related to IT, stuff tends to live more or less forever. Thus, appliances/integrated systems/embedded in the form of self installing images, ready to run solutions as virtual images as well as hardware appliance have their place and will continue to have a place going forward. However, what is really taking off as far as market growth is concerned is the Public Cloud, IaaS (Infrastructure as a service), as well as PaaS (Platform as a Service), and SaaS (Software as a service) along with the Marketplaces offered by the Public Cloud Service Providers (CSP) . In the appliance model one concern has always been how to handle the OS that is bundled with the application(s) into the appliance. What to do with security updates and how to manage the implied responsibility for the OS by the one providing the appliance. In the solution oriented way offered by the super market approach of Amazon and other CSPs many of these concerns go away thus reducing a business/support problem to a purely technical problem. Technical problems are often much easier to solve and this is really what the rest of this blog is all about. Before we get there however, lets take a little closer look at the support and update issue outside the public cloud to help set the stage for what is to come.

As an embedded systems vendor there is an implied responsibility for the OS that gets shipped along with the appliance. In the light of hartbleed, shellshock, and other issues this may appear as a daunting task, even if you get backed by SUSE as the distribution vendor. On the SUSE side we tried to make this reasonabaly easy with the SUSE Lifecycle Management Server. But this solution was not really sufficiently universal and it expected the provider to get out of their comfort zone to make updates available for the underlying OS via a web infrastructure. For many the operation of such a service is way outside their core business and way outside their focus area, no matter how big of a check the end customer is wiling to write. In the Public Cloud, and I will focus on Amazon Web Services from here on forward these concerns are eliminated.

With the semi-technical background behind us lets take a closer look at how it works. First lets address the “how does stuff get updated” question. SUSE maintains an update infrasturcture within Amazon EC2, details about how this works are included in this presentation from SUSECon 14, that any on demand image in the Amazon EC2 infrastructure has access to. Thus as a solution provider the question about how to update the underlying OS for security fixes is resolved. “All” that needs to be done is to offer a solution in Amazon Marketplace that is build on top of a SUSE on demand image. On demand SUSE Linux Enterprise Server images automatically connect to the SUSE maintained update infrastructure and thus have access to security and bug fixes. Such solutions can be offered in the Amazon Marketplace or the general Amazon EC2 catalog. There are 3 ways to get there:

1.) build your image with KIWI,
2.) build your image with SUSE Studio
3.) create a new image from a running customized instance.

1.) Build your image with KIWI

KIWI is an image build system that allows you to build images in numerous form factors. A setup guide for using kiwi on a SUSE Linux enterprise Server 12 system can be found here  and for SUSE Linux Enterprise Server 11 here . The examples provided by the kiwi-doc package are a great place to start. The Cloud:Images project in the openSUSE build service contains images built with openSUSE as sub-projects for a number of public clouds. The build set up for openSUSE and SLES is pretty much the same with the key difference being the repository configuration file in the kiwi image description. Building images for Amazon EC2 requires the following type definition in the kiwi configuration file:

<type image=”vmx” filesystem=”ext4″ boot=”vmxboot/suse-SLES12″ bootloader=”grub2″ kernelcmdline=”console=ttyS0,115200n8 multipath=off net.ifnames=0 NON_PERSISTENT_DEVICE_NAMES=1″ boottimeout=”1″ installprovidefailsafe=”false” firmware=”ec2hvm”>

In this case we are building a SLES 12 image that is targeted for an HVM image. Amazon prefers that new images use HVM and are SSD backed. The other piece that belongs into the kiwi configuration file that makes things work is the Xen kernel module which must also be included in the initrd that kiwi builds. This happens with the following specification in the <packages> section:

<package name=”xen-kmp-default” bootinclude=”true”/>

You also need to specify the bootloader xen package:

<package name=”grub2-x86_64-xen” bootinclude=”true”/>

The openSUSE example in the openSUSE build service is a relatively minimalistic setup and thus you can use it to build up from there. For images that are destined for the Marketplace Amazon requires that the default user is a user other than root. This is accomplished via the cloud-init config file that you will want to create and place into the kiwi overlay tree. There are also interesting tidbits to be found in the config.sh file. Most of the settings should apply to your image build.

In order to resolve the update problems of appliances discussed earlier you want to include the following packages in your build:

<package name=”cloud-regionsrv-client”/>
<package name=”regionServiceClientConfigEC2″/>
<package name=”regionsrv-certs”/>
<package name=”regionServiceCertsEC2″/>

These packages contain the necessary functionality to hook into the SUSE operated update infrastructure in Amazon EC2. You will need to enable the registration service which is accomplished by adding

suseInsertService guestregister

to the config.sh file. Your image must also contain at least the release package for SUSE Linux Enterprise Server. On demand images published by SUSE contain the release packages of all modules to allow customers access to all the packages in SLE 12 that are considered part of SLES. This looks as follows in the kiwi configuration file:

<package name=”sle-module-adv-systems-management-release”/>
<package name=”sle-module-adv-systems-management-release-POOL”/>
<package name=”sle-module-legacy-release”/>
<package name=”sle-module-legacy-release-POOL”/>
<package name=”sle-module-public-cloud-release”/>
<package name=”sle-module-public-cloud-release-POOL”/>
<package name=”sle-module-toolchain-release”/>
<package name=”sle-module-toolchain-release-POOL”/>
<package name=”sle-module-web-scripting-release”/>
<package name=”sle-module-web-scripting-release-POOL”/>
<package name=”sle-sdk-release”/>
<package name=”sle-sdk-release-POOL”/>
<package name=”sles-release”/>
<package name=”sles-release-POOL”/>

You can pick and choose which modules you want to enable in your appliance. We strongly recommend to at least enable the public cloud module in addition to the base product.

If you already have access to the SUSE Linux Enterprise repositories (remember access for development purposes if free through Partner Net) you probably want to set up a local SMT server to decrease you kiwi image build times. Optionally you can build your image on a SLES instance running in Amazon and pull the updates from the SMT server that SUSE provides to update on demand images. With this you are well on your way to build a SLES image that can include your application(s) that you want to offer in Amazon Marketplace. Once the image is built you want to upload it as an on demand image. This process is automated with the ec2uploadimg utility. As the base image to help with the upload you want to use a SLES 12 on demand image for the region where you want to offer your solution. Find the latest SLES 12 image with pint using the following command.

pint amazon images –region THE_REGION_OF_INTEREST –active –filter ‘name~sles-12’ | grep -v byos

Use the ami-id of the entry in the configuration file for ec2uploadimg and then use the –root-swap option. Once your image is uploaded you have an on demand image that can connect to the SUSE update infrastructure in Amazon EC2. Once tested and ready to publish you can copy the image into different regions using the

aws ec2 copy-image

command. The magic that turns the image into an on demand image will come along in the copy image process. With the image uploaded to your account you can publish it in the Amazon Marketplace and/or in the general catalog. For publishing to the general catalog you can utilize the ec2publishimg utility.

This is pretty much all that needs to be done to build an image with kiwi and publish it in the Amazon Marketplace. You do not need to have a contract with SUSE or any kind of formal business relationship. You do need a Marketplace agreement with Amazon. Thus all the steps that may have been a hindrance in the integrated system approach outside of the public cloud have been resolved.

2.) Build your images with SUSE Studio

SUSE Studio is a web application that uses KIWI in the backend to build images. As such it has all the magic, spare the user configuration for cloud-init, built in. You can provide the cloud.cfg file as an overlay file in SUSE Studio and build your image. When using the upload functionality built into SUSE Studio for Amazon you automatically get an on demand image that is ready to tie into the SUSE operated update infrastructure in Amazon EC2. The caveat is that when you build an image with SUSE Studio it will be a PV image that is backed by standard, i.e. spinning disks, storage. Thus this image will launch a bit slower than an SSD backed image. With SUSE Studio you can get your image built in 15 minutes and be up and running in Amazon with full access to update repositories in no time.

3.) Create a new image from a running customized instance

Start out by running a SUSE Linux Enterprise on demand image. You can find the latest image using pint as shown previously or you can launch it from the quick launcher. Once the instance is running login as ec2-user. You can use “sudo -i” to become super user and install software as you see fit. SLES packages are available from the configured repositories.

If you want the default user to be a different user than “ec2-user” modify the cloud-init config file (cloud.cfg in /etc/cloud). For a customized message of the day replace the /etc/motd file. After you have customized the running instance to your liking it is time for some clean up before you create a new image. You need to follow these steps to ensure the user of your Marketplace image has a first use experience.

1.) Reset the repository registration
rm /etc/zypp/repos.d/*
rm /etc/zypp/credentials.d/*
rm /etc/zypp/services.d/*
rm /var/log/cloudregister

2.) Reset the instance configuration
rm -rf /var/lib/cloud/*

3.) Reset the region association
Remove the comment and the entry that refer to smt from /etc/hosts

4.) remove the ec2-user
from /etc/passwd
from /etc/shadow
sed -i ‘s/ec2-user//’ /etc/group
rm -rf /home/ec2-user/

With the reset complete, stop the instance (shutdown -h now) or from the AWS web console

Now you are ready to create a new image

aws –profile PROFILE_NAME ec2 create-image –instance-id ID_OF_STOPPED_INSTANCE –name NAME_FOR_YOUR_IMAGE –description DESCRIPTION_FOR_YOUR_IMAGE –block-device-mappings ‘[{“DeviceName”:”/dev/sda1″,”Ebs”:{“VolumeSize”:10,”VolumeType”:”gp2″}}]’

Note the image creation command above is for HVM images, the preferred image type. For a PV image the DeviceName value needs to be set to “/dev/sda“.

The new image is ready for the Amazon Marketplace and can be copied to any region. It is an on demand image and will register with the SUSE maintained update infrastructure.