The European Space Operations Center (ESOC) within the European Space Agency has chosen SUSE Linux Enterprise Server as its operating system in its deep space ground stations and spacecraft operational control center.

The Challenge

The European Space Operations Center (ESOC) in Darmstadt, Germany, is respon­sible for monitoring and controlling space­craft throughout their operational life. ESOC also sets up, operates and maintains spacecraft-­support infrastructure on the ground, including the network of ground stations in Argentina, Australia, Belgium, French Guiana, Portugal, Spain and Swe­den (ESTRACK). The network must provide cost-­effective, accurate and timely communications with manned and robotic missions that may be billions of kilo­meters from the Earth. It uses cutting-­edge technology including high-­precision point­ing antennas, multi-­frequency radio recep­tion and transmission, cryogenically cooled low-­noise amplifiers and high-­power uplink amplifiers.

ESOC must also ensure consistent and re­liable operations throughout the full dura­tion of ESA missions that can span decades. This presents significant challenges in a number of areas, not least the technology platforms that support communication with and control of satellites and other spacecraft. The organization undertakes continuous research to determine which technologies will offer the best long-­term maintainability.

Ernesto Doelling, head of the ICT Evolution section at ESOC, said: “Performance is important, but it’s the reliability and the maintainability that are key for the ESOC Missions Operations Infrastructure when it comes to operating systems. From an engineering perspective, we need a product that is easy to understand and that isn’t overcomplicated. And it’s vital to choose a system that will be backed by long-­term professional support.”

Depending on each mission’s individual requirements, ESOC had previously deployed a variety of different operating systems, including Microsoft Windows NT, OpenVMS, Sun Solaris and other variants of UNIX.

“There are always unforeseen things in any new software deployment, but the consistency and control that SUSE Linux Enterprise Server gives us means we can rule many things out very quickly.”

SUSE Solution

Aiming to rationalize its IT landscape as far as possible, and considering factors such as usability, long-­term maintainabil­ity, and the availability of professional external support, ESOC decided to adopt SUSE Linux Enterprise Server 11 as the operating system for its network of ground stations.

“Our internal IT function recommended Linux—and specifically SUSE Linux Enter­prise Server—as the most reliable option,” said Euan Hogg, Senior IT engineer at the European Space Operations Center.

ESA has deployed around 100 instances of SUSE Linux Enterprise Server within the last 12 months, including both new sys­tems and migrated environments. How­ever, some applications have remained on their legacy platforms.

“Most commonly, we migrate—but we eval­uate on a case-­by-­case basis,” said Marco Lanucara, Head of the Systems and Project Support Section in the ground stations division. “For example, we determined that for a certain application running on QNX, the link between the OS and the hardware made it better not to migrate.”

The deployment of SUSE Linux Enterprise Server in the ground stations aligns them to the ESA baseline operating systems. Indeed, SUSE Linux Enterprise Server is the baseline for other ESA major applications, including the Mission Control Systems and the Simulator. In the future, although the hardware platform may vary from domain to domain, there will be a single version of Linux for all ESA applications using that operating system. Any new release or service pack will be validated at ESA, then uniformly used and deployed in all domains, including the ground stations. This was not the case in the past, when different approaches were used in different domains.

ESA generally uses specialized applications custom-­developed by external software companies. The porting to Linux is also handled by external companies—though not necessarily the same ones that wrote the original software. To increase maintainability and reduce risk, ESA has a policy of obtaining sufficient intellectual property rights for each piece of software to enable it to outsource future development to third parties.

“It reduces our risk in the long term if we know that two or three companies have done some work on the software, and if we have the source code,” said Marco Lanucara.

While ESOC has encountered no major issues with deploying or using SUSE Linux Enterprise Server, there were some minor challenges to overcome during the migra­tion of the telemetry/telecommand appli­cation from Solaris to SUSE Linux Enterprise Server. “The telemetry and telecommand processor is perhaps the most critical application we have in the ground stations,” said Euan Hogg. “Telemetry is the data received from the satellite, while telecommand is the data we send to the satellite to operate it. Our colleagues in the software engineering division ported the application from Solaris to SUSE Linux Enterprise Server to keep it maintainable from an operating system point of view.”

The key challenge ESOC faced was getting the telemetry/telecommand application to work with the legacy network infra­structure. “The system previously used six Ethernet ports, and there was all sorts of routing magic happening in the Solaris ker­nel to get the IP packets to go in the right direction and with the right quality of service,” said Euan Hogg. “When we first migrated the application to SUSE Linux Enterprise Server, it worked perfectly, but we then decided to reduce the six Ethernet ports to two. The Solaris kernel behaves in a different way from the Linux kernel in routing packets, and it took some time and effort to fix. But that’s really the only issue we encountered related to the porting—and we solved it!”

The Results

For us, the key value in using SUSE Linux Enterprise Server lies more in what surrounds the system than in any specific functions,” said Euan Hogg. “It’s the ability to get the necessary patches, fixes and updates in a timely manner, and it’s the visibility of software licenses. No configuration choices are forced upon you, and the system is very logical and well-organized at the command line if you come from a traditional UNIX background.”

With SUSE Linux Enterprise Server as its baseline OS for ground stations and mission control, ESOC has a reliable solution backed both by the open source community and by SUSE. This has given the organ­ization confidence that the software will provide long-term maintainability—one of its key objectives.

“I’ve worked with open source since Slack­ware v.3 back in 1995,” said Euan Hogg. “There’s a huge community interested in open source products, and the industry over time has produced serious products worthy of being called enterprise-level. We are confident that we can rely on SUSE, and that our users can rely on us as a result. A key factor in our choice of SUSE Linux Enterprise Server was its combination of open source backed by enterprise-class support. As good as it is, we couldn’t rely solely on the Linux community for something as important as the objectives of ESA.”

ESOC introduced the first Linux baseline for Mission Control systems eight years ago, during the preparation of the Ground Segment for Herschel and Planck, the first ESA missions in orbit controlled by SUSE Linux Enterprise Server. “Since the introduction of SUSE Linux Enterprise Server 9 in 2004, ESOC has successfully kept its Linux baseline up to date with SUSE Ser­vice Packs, and ultimately migrated to ver­sion 11 in 2011, ensuring continuity for both the Operational and Development/Engi­neering communities,” said Ernesto Doelling.

The adoption of SUSE Linux as the current baseline at ESOC is making it easier to achieve standardization and consistency. ESOC uses YaST® and AutoYaST to create, maintain and automatically deploy differ­ent configurations of SUSE Linux Enterprise Server for different requirements. This enables the organization to ensure that can deliver additional or replacement servers for existing missions that exactly match the existing or outgoing hardware.

“One of the key benefits YaST gives us is trust from the people we deal with,” said Euan Hogg. “The end-user has to be confi­dent that he can get an exact replacement for what he had three years ago, and we can guarantee that now. YaST also gives us the option to do version control by using XML files, and we can then respond to requests to add, remove or configure applications through that mechanism—it’s excellent. In my experience, Kickstart in Red Hat Linux was a little less intuitive and accurate. YaST makes life extremely straightforward, but also it doesn’t completely hide everything from you. I’m not often impressed by operating systems, but the whole AutoYaST system makes SUSE Linux a very powerful environment.”

Equally, the move to SUSE Linux saves time and effort when it comes to deploying new or updated systems, ultimately reducing costs and improving quality for ESOC.

“Installation times are faster with Auto­YaST, without a shadow of a doubt, and we have confidence that what we deliver is consistently the same every single time,” said Euan Hogg. “There are always unfore­seen things in any new software deployment, but the consistency and control that SUSE Linux gives us means we can rule many things out very quickly.”

While ESOC has substantial Linux skills in-house, it appreciates the ability to call on professional support from SUSE to short­cut troubleshooting exercises. “The SUSE engineers are brilliant,” said Euan Hogg. “We used them for a problem where we began to lose the GUIs for certain appli­cations, which continued to run without problems. The SUSE engineers examined the system dumps, and rapidly pinpointed it as a problem with sound—which we don’t really use in any case. That would have taken me so long to find out! Speaking as a technical person, having that quality of support is a key part of choosing a platform.”

With SUSE Linux Enterprise Server powering a significant number of its ground segment systems, ESOC has greater confidence in its ability to offer seamless, reliable operations.

“Our satellites are protected by multiple redundant systems, so any fault with the operating system of a ground computer wouldn’t have a catastrophic impact,” said Marco Lanucara. “But it’s certainly true to say that the reliability of the OS baseline plays a crucial role in ensuring the smooth and efficient operation of ESA’s satellites, supporting the important scientific work they accomplish.”