Hypergiant Industries focuses on solving humanity’s most challenging problems by delivering best in class artificial intelligence solutions and products. The company creates emerging AI-driven technologies for Fortune 500 and government clients working in a host of sectors, including space science and exploration, satellite communications, aviation, defense, health care, transportation and more.
Previously a Cyber Transport Journeyman in the U.S. Air National Guard, Bren Briggs is used to being deployed to the most hostile environments to design and run command and control communications. Now responsible for DevOps and Cybersecurity at Hypergiant, Briggs draws on his military background, bringing expertise in designing and building repeatable, secure deployments of Kubernetes and other infrastructure to support Hypergiant’s AI and machine learning (ML) applications and customers.
Briggs is one member of a wider team of experts working for the U.S. Department of Defense’s official DevSecOps enterprise services platform, DOD PlatformONE. Briggs works alongside SUSE RGS’s co-founder and Kubernetes Edge Architect Chris Nuber, and Hypergiant’s Space Systems Architect Chris Tacke. Together, they are on track to achieve the impossible in the most hostile environment of all—putting K3s clusters to work in orbit on military satellites for the very first time.
Leveraging Rancher Prime’s lightweight and compact Kubernetes solution, K3s, Hypergiant aims to deploy the modern, lightweight software systems that will drive the next evolution and commoditization of the satellite industry.
The Journey to Kubernetes
Hypergiant and SUSE RGS are working together with DOD PlatformONE to demonstrate the benefits of DevSecOps, Kubernetes and AI/ML apps in remote and often disconnected environments. Working with K3s (recently accepted as a CNCF Sandbox project), the team is developing and integrating their software pipeline with the EdgeONE and SatelliteONE missions, including the launch of Hypergiant’s Kubernetes-embedded satellite planned for the first quarter of 2021.
Briggs started working with Kubernetes several years ago to solve the problem of managing multiple Docker containers. At that time, containers were built and shipped like virtual machines (VMs) — updated and managed in Puppet. With a growing need for consistency, Kubernetes became an important way to reliably network the estate of containers and manage them centrally.
In 2019, Briggs brought this experience to Hypergiant, where he noticed repetition in existing development methods. He recognized the need for a platform that would allow developers to deploy repeatable patterns with less manual effort. They needed a cloud that could run anywhere, and so Briggs suggested Kubernetes as a way to automate the creation of repeatable workloads. In early 2020, the team started rolling out Kubernetes on several internal systems. As they ran on AWS, EKS was initially deployed to orchestrate the first cluster, but soon, K3s entered the team’s consciousness.
With a mandate to find a way to run Kubernetes in space, Briggs, Tacke and Nuber began investigating. Briggs was impressed. K3s was designed as a lightweight, compact version of Kubernetes that could run in remote, low-power, hostile environments. Available ‘off the shelf,’ K3s was easy to install and avoided the need for Briggs and the team to ‘roll their own’ lightweight distribution. Crucially, Briggs estimates deploying a cluster takes just a day using K3s, compared to weeks in a manual methodology. The cluster itself builds in 10 minutes using Ansible plays. Provisioning base services takes another 30 minutes. Production can be ready before launch once the automation has been written.
There isn’t another version of Kubernetes that can meet our use case. K3s has allowed us to build infrastructure suitable for use in space without the need to create our edge distribution.
A World First: Putting Kubernetes Clusters in Orbit
The satellite industry faces several problems. Firstly, space-rated hardware is costly. Software development and delivery processes are slow. And it’s cold — really cold. On-orbit satellite software updates are often not possible or incredibly time-consuming and expensive. As a result, AI/ML capabilities are far behind those currently available on earth.
Satellite connectivity and bandwidth are poor, which makes downloading large images and other data difficult. The SatelliteONE mission has been designed to solve this problem. The project will demonstrate DevSecOps in space by leveraging PlatformONE’s CI/CD pipeline alongside Kubernetes provisioning and deployment. Importantly, it will evaluate the use of lower-cost hardware on satellite payloads, show how the rapid delivery of software updates in space can be done and demonstrate the use of AI/ML software in orbit.
The U.S. Department of Defense, along with governments worldwide, are looking for ways to build longevity and sustainability into satellite fleets. They want to modernize their entire fleets to be managed, maintained, and, crucially, reconfigured for various use cases for the long term. This is a massive shift, but investment in modernization will create agile technology stacks that can be more easily updated and replaced in the future.
Because K3s is packaged as a single <40MB binary, it reduces the dependencies and steps needed to install, run and auto-update a production Kubernetes cluster. Supporting both ARM64 and ARMv7, K3s works just as well on a Raspberry Pi as it does with an AWS a1.4xlarge 32GiB server. K3s is enabling Hypergiant to deploy the modern, lightweight software systems that will drive the next evolution and commoditization of the satellite industry.
Deploying K3s to Achieve the Impossible
SatelliteONE’s initial mission is an important one — to take a picture of Baby Yoda. While this doesn’t sound particularly groundbreaking, this exercise’s primary purpose is to demonstrate the ability to capture images and perform AI and ML workloads at the edge.
This is an important milestone given the bandwidth constraints experienced in space. When dealing with a link, measured in kilobits per second, only available for a few minutes at a time during a flyover window, transfer efficiency is the ultimate goal. In this use case, the team wants to transfer the minimum amount of information.
The architecture consists of two Raspberry Pi server nodes, independent of each other, with one being a warm backup. There are two Raspberry Pi 4 worker nodes, each loaded with a Pi camera, and another Raspberry Pi 4 with an accelerometer and light sensor. Finally, there is a Jetson nano module (developed by NVIDIA), a $99 IOT AI processor mounted on the satellite and used to capture and process the image of Baby Yoda.
The two master nodes have an additional ethernet adapter attached to another switch and have connectivity to the Cygnus device and external network, allowing communication with the ground (Cygnus is the vehicle the team is hitching a ride on courtesy of Northrop Grumman). There are two independent K3s server nodes in place. This is critical should one server device fail in orbit — the team can quickly communicate with the server node, automatically failover, or run scripts to rebuild the cluster in a matter of minutes. Since the clusters are logically separated, all the K3s images and the application workloads are stored locally on each node — less overhead and one less point of failure.
Importantly, this is about sending analysis rather than the raw image itself. Ultimately, a human can still make decisions as to which photos to downlink. However, the focus here is to process decisions at the edge, sending analysis and results instead of the raw dataset.
The team is also conducting a temperature experiment to observe how the Jetson behaves in a vacuum under a heavy computational load. Why, though, would the team put a whole satellite in space to take a few pictures of Baby Yoda and conduct temperature measurements?
By proving these basic processes work, the team can highlight the use case for Kubernetes in orbit. Industries of all kinds that rely on satellite communications and imaging want to create modern satellite fleets that can be used for longer and have less environmental impact.
By deploying novel compute technologies, like K3s on orbiting devices, their longevity and ongoing usability are assured. Working with K3s, Hypergiant, SUSE RGS and DOD PlatformONE are creating a commercial future for data processing in space.