To improve match analysis, gain a better understanding of patterns in games, and enable its clients to translate that insight into winning betting strategies, Smartodds is always looking for ways to improve its analytical and predictive big data models. With a business model that relies entirely on IT, Smartodds is very eager to take advantage of the latest technologies, tools, and solutions on the market to improve its analytics capabilities, increase the likelihood of success for its clients, and maintain its competitive edge.
David Mace, Senior System Architect at Smartodds, said: “We started out analyzing just the UK football leagues, and now we manage complex statistical models for leagues all over the world. This means that data volumes and the number of statistical models are always increasing. In addition, because we analyze activities around the world, 24/7 availability is essential to keep business operations running smoothly. Avoiding a single point of failure and improving resiliency is therefore at the heart of all our IT infrastructure decisions.
“In recent years, we’ve also gone from being a football-only shop to offering statistical analysis for all kinds of sports, including rugby, tennis and golf. To ease expansion into new markets, we’re also taking up sports that are popular in other geographies, like the National Hockey League in the US and Canada.”
With the business extending its service portfolio, Smartodds needs to run more and more statistical models reliably—without letting its operating costs spiral out of control and affect its profit margins.
Liam McCafferty, 3rd Line Engineer at Smartodds, summed up the operational difficulties: “Our revenue is entirely data-driven. As a result, even with a small team, we can get our hands on quite advanced solutions that help our data scientists and quantitative analysts dig deeper into exploding data volumes. But using the latest technology comes at a cost—our job is to balance innovation with efficiency.
“Our goal is to push our data through more sophisticated models faster, and to generate more accurate predictive insights. We can only achieve this cost-efficiently by finding smarter ways to utilize our computing capacity and to balance our diverse workloads across our servers, delivering maximum performance when and where it is needed, without increasing management complexity.”
When Linux containers started becoming popular among developers, Smartodds was quick to adapt its processes to embrace container technology. Liam McCafferty elaborated: “For us, containers are an essential technology. They enable us to consolidate our workloads onto fewer physical machines. At the same time, containers enable us to increase reliability, as we can now afford to run at least two instances of our internal, backend API services behind load balancers instead of relying on just a single virtual machine.”
Furthermore, adopting containers also simplified continuous integration (CI) and continuous deployment (CD) workflows at Smartodds, increasing deployment frequency and cutting time-to-market for new features. The team is using GitLab, a leading open source solution for the entire software development lifecycle, to manage its code throughout the full lifecycle. Using the flexible webhooks feature of its private GitLab instance, Smartodds integrated the tool with the Jenkins automation server, also open source, enabling its developers to trigger integration tests and even deployments via a single Git commit.
David Mace commented: “Many of our basic API services were already following a microservices approach, so focusing on DevOps and making it easier for our developers to iterate and deploy their independent services more quickly was just the logical next step. Linux containers allowed us to empower developers by automating the tedious and time-consuming tasks around testing and deployment.”
Liam McCafferty added: “The standardization of deployment environments also meant that essentially the exact same stack would run in development on the local machine, in the test environment and the development environment, reducing deployment issues and making the whole team more agile. As a result, we can ship new features and insights faster to our clients, which helps them make informed betting decisions and increase their odds of a good return.”
DEPLOYING AN ENTERPRISE-CLASS CONTAINER MANAGEMENT SOLUTION
Moving to Linux containers already addressed many of the growth and management challenges Smartodds had been facing, but running around 20 applications reliably still required a significant amount of manual work and constant monitoring.
David Mace remarked: “We started with a custom-built solution to operate our growing number of Linux containers. At a certain point, we realized that we would be much better off using SUSE CaaS Platform. The SUSE solution is an off-the-shelf, fully supported product, which offered much more stability than our fragile existing system. A colleague, who has since left the company, had developed it in-house and left no comprehensive documentation behind.”
To reduce the system administration workload for the small in-house Linux team, Smartodds had already standardized its Linux servers on SUSE Linux Enterprise Server in combination with SUSE Manager. David Mace recalls: “Simplifying our Linux environment by moving everything to SUSE Linux Enterprise Server made it much easier for us to support all the different microservices. We are more flexible and productive as a result. Standardizing on SUSE Linux Enterprise Server also facilitated knowledge sharing and established best practices across all Linux systems. Centralizing basic administration with SUSE Manager was instrumental to our success, helping us to cope with our growing IT environment.”
For a smooth development experience where software engineers can run their applications in an environment that closely resembles the production environment, without requiring complicated setup and configuration, around 30 developers at Smartodds also use SUSE Linux Enterprise Desktop.
Smartodds followed the same logic when evaluating SUSE CaaS Platform. The team quickly realized that setting up and operating the open source container orchestration solution Kubernetes would be quite time-consuming, and without extensive experience it would be difficult to get the right configuration for optimal performance. Liam McCafferty noted: “SUSE CaaS Platform is very easy to set up, so we could focus on our business challenges instead of basic configuration tasks.
“We were also happy to learn that SUSE uses the Salt Open configuration and automation toolset to configure the SUSE CaaS Platform. Having prior experience with Salt from our other SUSE systems helps us understand the nitty-gritty details of the automated processes, which will help if we ever need to look under the covers.”
CHOOSING THE RIGHT MIGRATION PATH
Smartodds plans to gradually move most of its Linux workloads to the SUSE CaaS Platform. The company set out three separate phases to focus on three different kinds of workloads.
As a first step, the team has already started migrating existing Linux containers over to SUSE MicroOS, a purpose-built operating system optimized to run the next generation of applications in Linux containers with a small footprint. At this stage, around 20 web applications written in Python using the Django framework, which provide basic APIs to backend services for customer-facing applications and also internal desktop applications, will be moved from the custom-built container environment to the standardized SUSE CaaS Platform.
SUSE CaaS Platform uses the Kubernetes orchestration solution, which adds a management layer that defines the smallest units of deployments, known as ‘pods’. Pods make deploying complex container configurations (in which several containers depend on each other) much easier as they can be managed and scaled as a single entity. However, during this first phase, Smartodds plans to take existing containerized applications and run only a single container within each pod.
David Mace explained: “As we already have a number of containerized applications, these will be the first that we deploy to the SUSE CaaS Platform. In the second phase, we are planning to move the actual models to the SUSE CaaS Platform. Our 20 or so statistical models are the very core of our business and usually run once per night. These are written in R, an open source software environment for statistical computing and modeling. Running our statistical models is highly resource-intensive, and we aim to set up two large physical servers as nodes for the Kubernetes cluster, which is managed by SUSE CaaS Platform.”
The statistical models generate the data that is delivered to internal and customer-facing applications through the (already containerized) API web applications. The statistical models are also where the number crunching happens, and the results, reports and insights are then written to MySQL databases.
Due to the complexity of the applications and the number of dependencies involved, the containers for the statistical models will be substantially larger than for the API microservices. Liam McCafferty added: “One of the main motivations for deploying our statistical models to the SUSE CaaS Platform will be the elimination of a single point of failure in our current architecture. Using the Kubernetes orchestration technology, we can take advantage of the integrated self-healing features to ensure that at least a de fined number of instances is always running. Thanks to this, we can minimize administration workload while maximizing availability—ensuring that the models run reliably, without the need for manual intervention if one model were to crash during batch processing.”
In the third migration phase, Smartodds plans to take systems that currently run as virtual machines on VMware, set the services up as applications within Linux containers, and then deploy the containers to the SUSE CaaS Platform. Systems that the company is considering moving to the SUSE CaaS Platform during this final migration phase include: the open source RabbitMQ message broker, the open source in-memory database Redis, and the open source collaboration and communication solution Rocket.Chat.
David Mace explained: “By deploying standard services like RabbitMQ, Redis and Rocket.Chat as containers, we can reduce management overhead further still and pack more systems on fewer physical servers, since we can also distribute the workload more flexibly across all SUSE CaaS Platform nodes. Once all three migration phases are complete, we expect to run around 300 containers or pods on SUSE CaaS Platform, including production pods and replicas. And that’s only the beginning.”
FLEXIBLE DEPLOYMENT INFRASTRUCTURE
Using physical resources efficiently is a key concern for Smartodds, so the company is glad that it can run Kubernetes nodes as virtual machines and as bare-metal systems within SUSE CaaS Platform. This flexibility allows Smartodds to deliver high performance with large amounts of memory through bare-metal nodes to run the compute-intensive statistical models. At the same time, the company can run its backend API microservice web applications on more cost-efficient virtual machines.
Liam McCafferty commented: “Being able to tailor system performance more closely to the business requirements, simply by adding labels when we deploy pods, will help us increase operational efficiency. This process will be fully automated, without increasing the system administration workload.”
Implementing SUSE CaaS Platform will deliver many benefits for the development team at Smartodds. The pre-packaged and fully supported solution replaces a range of bash scripts with a standardized and declarative way to de ne deployments directly in the Git source code repositories using the simple YAML markup language.
David Mace added: “Using SUSE CaaS Platform, we can streamline our DevOps workflows substantially and benefit from the Infrastructure as Code approach. Controlling deployments has become much easier, so we can now offer a state-of-the-art continuous integration and deployment solution to our developers, helping them to get new features to clients faster.”
Liam McCafferty confirmed: “Some developers were amazed to see how quickly and simply they can now deploy applications to the SUSE CaaS Platform. Before, deployments would take 30 minutes. Thanks to SUSE CaaS Platform, developers can now get new releases online in about one minute—up to thirty times faster than before. And the deployment time includes replicas to ensure high-availability and increase performance if needed. This saves the team a huge amount of time and boosts developer productivity even further.”
The whole process has been simplified with SUSE CaaS Platform. In the past, container images were pre-built by the developers, and usually never updated. With SUSE CaaS Platform and the integration of the container registry and containers into SUSE Manager, the infrastructure team can keep the container images up to date. With this central control, the company can make sure that all container images are updated and patched alongside all other systems, improving security across all applications.
Smartodds also uses the SUSE open source tool Portus, which provides a sleek, modern user interface for the container registry. In addition, Portus introduces a granular authorization service to restrict access to container images to only those developers and operations employees who need it.
SUSE CaaS Platform includes more security features, for example to help developers keep credentials, passwords, and other configuration values safe—outside of the actual application code and the deployment specifications.
For Smartodds, installing SUSE CaaS Platform was a big time saver. Liam McCafferty added: “We had looked at Kubernetes before—it is truly remarkable how quick and easy the installation of the SUSE CaaS Platform was. In the end, it was cheaper for us to buy and deploy SUSE CaaS Platform, than to configure Kubernetes and all its components manually from scratch. Using the pre-packaged product from SUSE definitely allowed us to get started with our migration faster.”
Today, Smartodds is already running a small SUSE CaaS Platform environment with 25 pods, running on three virtual machine nodes within a Kubernetes cluster, to prepare for the production migration of the first Linux containers. Liam McCafferty continued: “We’re already seeing the benefits of SUSE CaaS Platform. With a supported solution and professional documentation, we can significantly simplify the management of our container platform and take our Infrastructure as Code practices to the next level.”
David Mace summed up: “One of the most important advantages for us in operating our different systems is the improved isolation of applications and models. Containers make dependency management so much easier. In the past, it was sometimes difficult to deploy complex models to our existing servers, because when one developer made changes to a particular model, it might require different releases of a certain dependency than another model running on the same server. Resolving such issues is error-prone and time-consuming. With Linux containers deployed in the SUSE CaaS Platform, there are no more dependency conflicts.”
Smartodds also gained the capability to run different applications together on a single node more flexibly than before. This is something the team also plans to take advantage of when it comes to more dynamic load management. Liam McCafferty elaborated: “With SUSE CaaS Platform, we can use Linux containers in combination with flexible auto-scaling policies, helping us to ensure optimal performance for all services, without the need for constant load monitoring and manual capacity scaling. Running more services on fewer systems reduces total operating costs, increases our profitability and improves our return on investment.
“This automated, policy-based approach to container management also helps us boost availability, thanks to the self-healing features of Kubernetes and the SUSE CaaS Platform.”
David Mace agreed: “The automated clustering of pods behind load balancers really helps us to avoid downtime and slow, manual restarts that can delay the availability of the latest data, potentially disrupting client services. To mitigate the risk of failure, we used to buy expensive, highly redundant server hardware. By moving the high-availability features from the hardware to the SUSE CaaS Platform, we can start buying more, cheaper servers to keep up with business growth and increase performance, without compromising on reliability.”
The close integration of SUSE CaaS Platform with SUSE Manager will further reduce administration costs. Following its standardization on the SUSE Linux Enterprise platform, the team already controls 168 systems with SUSE Manager, and will add all its container images and containers once the migration has been completed.
One of the reasons Smartodds selected SUSE CaaS Platform was because the company already knew it could rely on SUSE as a partner. Dave Mace highlighted: “We have always been highly satisfied with the stability and performance of the SUSE Linux Enterprise platform, so it was an easy decision for us to turn to SUSE when we wanted an enterprise-class and production-ready container-orchestration solution. Working with SUSE reduced the adoption risk, not only because of the high product quality, but also because we knew that we could talk to highly competent support staff whenever we need to.”
Since implementing SUSE CaaS Platform, Smartodds has enjoyed the close collaboration with the SUSE team. Liam McCafferty recalls: “Right from the get-go, the team at SUSE took our feedback seriously and acted swiftly. As an early customer, we have helped to improve the user experience both for us and for other SUSE customers. It is truly encouraging to see how SUSE listens to its clients and takes their feedback on board.”
To support the production implementation, Smartodds will work directly with SUSE Consulting. Together, the team will optimize the company’s environment to achieve the best performance and facilitate a smooth migration to the new platform.
Together with SUSE, Smartodds is also evaluating ways to streamline its application portfolio to achieve the best performance and to maximize support coverage from SUSE for its mission-critical solutions.
LOOKING TO THE FUTURE
Currently, Smartodds runs all infrastructure on-premises. While a hybrid cloud approach is not a good match for the company’s business and IT requirements, flexibility without vendor lock-in was a crucial factor when Smartodds selected SUSE CaaS Platform. Since so many server vendors and cloud providers support SUSE CaaS Platform, Smartodds is free to choose the most cost-efficient and secure infrastructure to run its mission-critical workloads.
David Mace concluded: “SUSE CaaS Platform is a massive help for our small operations team. Thanks to the advanced container management solution from SUSE, we were able to reduce the setup time and rapidly deliver benefits to our developers. With faster, more reliable, fully automated deployments, our development team is no longer held back by rigid operational processes. Instead, they can iterate more quickly, add new features more easily, and release them to clients faster, without having to worry about reliability. And our operations team still has full control over the infrastructure and the systems running on it. We’re already reaping the rewards of SUSE CaaS Platform, and we are very excited to see how else it can help us to improve operations.”