The Public Cloud is growing rapidly. This means more providers with different image types, requirements and backends. With all the different backends and API’s how do you ensure custom images are in working order? Currently to do this testing you have to learn a new API for each provider or manually walk through each web console. This can be very time consuming. Also, in order to ramp up with a new provider you must learn a whole new set of commands. On top of that the process is manual which means it doesn’t scale, it’s not repeatable, and it’s error prone. So how do you automate and simplify this process?
Enter IPA or Image Proofing App. This is a command line tool that provides an image testing framework for Public Cloud images. With IPA you can now test custom images in a provider agnostic way with one tool and one API. In the first release, IPA supports the openSUSE and SLES distributions. It also supports the three largest cloud providers (AWS, Azure and GCE). However, it is intended to be distribution agnostic and framework transparent so both are easily extensible.
For each distribution there are specific synchronization points that must be provided. These currently include soft reboot and system update. The synch points not only test functionality but also act as dividers to separate distinct sections of a test suite. For example you can run a test to ensure the proper repos exist before and after a system update. The system update synch point will guarantee the order of tests. Speaking of tests, if you’re already familiar with Pytest conventions there’s no need to learn a whole new unit testing framework. IPA is written in Python and leverages the Pytest framework through Testinfra. For more information on writing tests see the Pytest docs and the Testinfra docs.
How to get started?
IPA is packaged and provided in the Cloud:Tools repo of the openSUSE build service:
zypper ar http://download.opensuse.org/repositories/Cloud:/Tools/<distribution> zypper refresh zypper in python3-ipa
And the provided SLES tests can be installed with:
zypper in python3-ipa-tests
The following example would run the test_sles suite against a SLES image in Amazon EC2:
ipa test -i ami-12334567 \ -a testaccount \ -d SLES \ EC2 test_sles
To get a list of available tests use the following command:
And to view the latest test results:
For more information on installing and using IPA see the getting started document in the GitHub repository.
Why create IPA?
The SUSE Public Cloud Team manages a growing number of images across the three major cloud providers (AWS, Azure and GCE). This makes it increasingly time consuming to build and test all of the images for a new release or for updates. The fact that testing has been relegated to a manual task takes time away from our continuous effort to improve our images and overall user experience . It also increases the chances for issues and bugs slipping through the cracks. IPA provides a much more efficient testing process, and allows us to test more images on more instances. Improving image quality and paving the way to release more frequently.
You might be wondering, “Why not leverage openQA? It’s a great tool for automated testing.” The main reason for a new tool is that we don’t want to teach openQA how to handle all the different public cloud backends. However, IPA can be integrated with openQA. Thus allowing openQA to test images launched in the Public Cloud.
IPA has been released as an open source tool/project under the GPLv3 license. Which means you can use it to test your own custom images and/or to validate the actual state of your instances (I.E. to unit test configuration management setup). If you have an image based on SLES or openSUSE, IPA provides a set of base tests that can be used. If you need a custom test, the base can be easily extended and/or new test directories can be included.