Let’s talk about AI

Artificial Intelligence. It’s everywhere. It’s changing our lives. We hear about it all the time, how it’s useful (or not). It raises many ethical questions, and we also hear daily about its misuses, about where it went wrong or how it is indeed helping us be better in our jobs. You can now find some form of AI feature/chatbot in almost any product or service around us.

But have you ever thought about HOW these AI services are actually implemented? Technically, an AI workload is a workload like any other, but it uses some special components, like GPUs and models. Most of the implementations we see are in the form of LLMs (Large Language Models), like chatbots,  or combined with other forms of generative content (like images, video and audio) and RAGs (Retrieval Augmented Generation) mechanisms.

 But it also needs data to train models, which are basically the brains of the operation, containing all the knowledge your application needs to know. A lot of data. Including data that’s supposed to be confidential, personal or strategic. Copyrighted data. Medical data. Military-grade information. Financial data. Stuff that really, really shouldn’t be in the public domain or be exposed in an insecure place or without proper supervision. And with new technology, we have new ways of getting information exposed, misused or exploited.

Unfortunately, that is happening a lot. In the wake of the AI wave, many solutions are being developed without taking into account basic security, reliability and infra-structure concerns. Simple measures like knowing where your components are coming from, applying security patches, monitoring your traffic. The complexity of the underlying technologies ( like Kubernetes and LLMs), plus the incredible variety of new projects that appear every day make it difficult to get things working in a reliable environment.

That’s where SUSE comes into the picture. You might have heard about the launch of  SUSE’s AI platform a few months ago. But what does it really do? How does it work? What is it for? How do I use it? Let’s address these questions, and by the end of this series, you should be able to understand the use cases, what each of its components do, and how to do a basic install with a working LLM chatbot!

What’s SUSE AI?

SUSE AI is a platform composed of a collection of cloud-native components that help address each part of a typical GenAI environment (or any other AI workload for that matter), in a secure and consistent way. It’s also about data sovereignty, as in, you are in control of the entire infrastructure and the data that’s coming in and out. The best-case scenario is to bring the infrastructure on-premises, and that’s where we can help you. No more leaving your critical data out in the open!

All the components are based off of SUSE’s own award-winning software solutions, made with care, excellent support and (a lot) of security processes in mind. We also thrive in making solutions that are open and without lock-in, meaning you always have choice. We provide the essential elements, including a secure OS, a secure Kubernetes platform, and a curated library of hardened AI components, so you can add your AI applications, models, and LLMs. We also provide tools to manage your clusters, get insights on troubleshooting problems, and secure communications within this environment. Let’s talk about each component, what it does, and how it contributes to our typical on-premises GenAI deployment. 

Let’s start at the bottom of the graphic: the infrastructure. This is composed of an operating system that is running on the bare metal, and an implementation of Kubernetes. Kubernetes is the gold standard for GenAI solutions due to its unprecedented advantages in relation to scalability and orchestration of workloads.

The OS layer

The bare metal needs an operating system to be used, SUSE has a long experience with OS development, and you might already know SUSE Linux Enterprise (SLE). It’s a family of Linux-based OSes with enterprise support. We’ll take a look at two of those products. There’s SLES, our general-purpose OS (read more about it here). It’s a Linux-based operating system that’s stable and secure by design, with a support lifecycle that goes up to 19 years (really!). Also, we’re very proud of our continuous improvements and many security certifications.

There’s also SLE Micro, which is a more compact, faster, immutable version of SLES that is even more secure. It inherits all the security features from SLES I’ve mentioned, and some additional ones, too.

Of course, we recommend using our OSes since we’d be able to support any issues you have with the OS itself, plus we provide and support the necessary NVIDIA drivers for your GPU. But you’re free to use other distributions supported by the next layer, which is Kubernetes. Don’t forget that SUSE has tools to manage almost any Linux distribution.

The Kubernetes Layer

Kubernetes is the gold standard for GenAI applications, and is widely used due to its flexibility, orchestration features, and scalability in cloud-native environments. Kubernetes also can become very complex when it scales, due to unique characteristics like the way networking works in it, which brings entirely new security aspects that need to be considered. It can also be difficult to keep everything supported and updated due to the continuous release cycles, and the sheer possibilities of customization.

SUSE offers two alternatives of Kubernetes distributions: RKE2 and K3s. Distributions, you say? Why would we need a Kubernetes distribution?

First, the complexity of implementing and maintaining Kubernetes by yourself is considerably lowered. Kubernetes has *a lot* of steps and components, which are updated frequently. Managing it all by yourself when deploying your clusters can be overwhelming. Having a curated distribution like RKE2 with secure and tested components makes installing a typical Kubernetes cluster as simple as running just a couple of commands per node, and waiting a couple of minutes. No hassles, it just works. And it runs on many supported OSes.  Plus, updates are as automatic as you’d expect, with minimum downtime. And, you have an entire ecosystem of integrated management tools that makes cluster administration as easy as interacting with a simple and efficient web-based interface. The main tool, Rancher Manager, also runs on many different environments, including but not limited to the Kubernetes layer provided by RKE2 itself and many, many cloud-provider specific implementations, like Amazon EKS, Google GKE/GCP and Azure AKS.

The AI Library

Most GenAI components and open-source AI projects are packaged as Helm Charts for deployment. It’s similar to the way software is packaged in the Linux world, as in you have repositories, and a main tool (helm) to install, uninstall and upgrade packages containing all the artifacts needed for a cloud-native Kubernetes application. These can be one or more deployments composed of various container images, which can come from many sources, along with proper versioning and the associated parameters and configuration elements needed for this application to be installed and deployed as expected.

Usually, these artifacts can be hosted on code repositories, like GitHub and GitLab, or, in one of many public or private Helm Chart repositories. This is where things can get muddy: where do each of those artifacts come from? Can you trust that each one has been verified for security issues, and that it indeed comes from a trusted source? Supply-chain attacks are common and can compromise any of the elements in a deployment with backdoors and code of questionable origin.

That’s where Application Collection comes in. That’s a curated collection of helm charts and container images that are curated by SUSE, where the origin and integrity of all artifacts are continuously monitored for vulnerabilities. Think of it as a secure place to get most of the common cloud-native open-source applications, but instead of getting them from many different sources, you get a sanitized, verified and secure version of them directly from SUSE. Oh yes, and it also has a growing SUSE AI stack as well.

The Security Layer

Running on Kubernetes brings some special challenges in regard to network monitoring and traffic analysis in the container world. Basically, there’s an entirely new structure of virtual networks, including internal DNS servers, and network contexts between all the pods and containers. And these are continually being created and destroyed, sometimes in seconds. How can you monitor such a moving target? Traditional tools just won’t work in this new world, so you are stuck with “black boxes” of data traffic. That’s where SUSE Security comes in. 

SUSE Security (née Neuvector) is able to monitor all Kubernetes cluster communications in network layers 2-3 and 7, automatically learn pod behaviors and create security policies, monitor/test for CVEs in real time in your container images. You can even monitor for processes, check for compliance and use it on your CI/CD pipeline! More details are available here.

The Observability Layer

Ah, the wonders of running micro-services. With this technology, you’re now able to leverage light containers for all your services, instantiated on-demand and orchestrated by our friend Kubernetes. And with multiple Kubernetes clusters, you now can run each component wherever you want or need, be it on-premise or on the cloud. But… what happens when you have a problem?

That’s where SUSE Observability steps in. In the micro-services world, you need to be aware of what we call the “4T data model”: Telemetry (metrics, events and logs), Tracing, Topology and Time. SUSE Observability continually ingests the first 2 items, correlates all that data to the Topology, and constructs a timeline where you can explore current and past data. It can also reason possible solutions for your issue and provide possible remediations. Read more about it here, or even better… play with a live environment yourself here!

What’s next?

In the next article, we’ll start setting up our SUSE AI Environment and take a look at the requirements and how to start setting everything up. Are you ready to explore SUSE AI?

Look out for part 2 of the series.