Following the introduction of APIs and growing levels of digitization across the economy, Experian experienced a huge rise in data volumes. This surge of data prompted the company to adapt its IT infrastructure to meet increased workloads. Experian deployed a SUSE software-defined storage solution, enabling easier capacity management, higher availability, and seamless data replication, giving it the power to continue delivering industry-leading insights 24/7.
KEEPING UP WITH THE DATA RUSH
The business intelligence business unit at Experian processes and analyses vast data sets from global sources, transforming this data into industry-leading business insights. To ensure that the insights Experian provides are timely and pertinent, it is essential that the company maintain an IT infrastructure with high reliability, strong availability, consistent performance and sufficient capacity.
Over recent years, Experian witnessed a rise in data volumes generated by demand for 24/7 services and the ever-increasing digitization of the world economy, both of which placed strain on the company’s IT infrastructure. Howard Samm, Head of Infrastructure for Business Intelligence at Experian, explains: “The technological age has given rise to the technologically savvy consumer. Businesses and individuals rightly expect companies to harness the power of data technology to deliver products and services around the clock. From the point of view of the business intelligence vertical at Experian, technological development has prompted a change in our clients’ expectations; they want even better, more comprehensive business insights. This change in demand had significant implications for us.
“In business intelligence, we undertake two main types of work. We run numerous data analytics-related batch processes, which deliver actionable insights to small and medium businesses, typically via a growing number of APIs, and we provide web-front ends for businesses to interact with our services. Both aspects of our work were impacted by a significant rise in data volumes. To put this into perspective, after incorporating new data feeds into our core databases to provide more analytical tools for our clients, we went from processing just a few GBs of data a month to hundreds of GBs an hour.”
From the outset, Experian saw great results running ZFS as a primary storage platform, which offered the flexibility for its systems to alternate between performance capacity and growth, and enabled them to adapt to changing client and internal business needs. However, Experian’s rapid pace of growth revealed shortcomings in this approach. To meet the changing demands of its clients, Experian required a more robust storage solution that would enable the company to safeguard performance levels and boost availability.
Howard Samm continues: “For a company in the business of providing quick data access, we need to be agile and responsive. We soon found that the surge in data volumes raised concerns around performance and storage capacity, and ultimately threatened to impact the quality of our services. “Overall, we faced three main technical challenges. First, our IT infrastructure was becoming increasingly difficult to manage. Our architecture was comprised of multiple manually integrated, standalone NAS [Network Attached Storage] devices.
These devices offered insufficient performance to match sudden peaks in demand. This is because we had to shuffle data stores between devices manually to free up space for other core processes, which often took days to complete and caused disruptions during switchovers. Moreover, if one of our NAS devices went down, it could potentially knock the whole system of offline which would cause significant business disruption.
“Second, we were operating with low levels of availability. In an earlier project, we established robust business continuity and disaster recovery systems, but at the expense of a significant degree of automation and responsiveness. As a result, our systems were becoming inflexible, and couldn’t effectively support the real-time access to data required by API enabled services.
“Third, we encountered issues with our asynchronous data replication processes. We were experiencing long delays caused by large-scale replications, which increased our exposure to potential data loss. We had reached the situation where, if a disaster event were to occur, we may have had to resynchronize our entire dataset, which would have put us at risk of losing core data.” To address these challenges, Experian implemented a solution from a third-party vendor in its storage environment. While this solution performed well, the new platform did not solve Experian’s performance and availability issues.
A STEADFAST SOLUTION FOR CHANGE
Experian turned to SUSE for help addressing the challenges posed by rapid growth. To provide stable performance in the short term, the company migrated its ZFS file system to SUSE Linux Enterprise Server.
“Initially, we implemented SUSE Linux Enterprise Server to support the performance of our ZFS environment and ease the pressure placed on the system by larger volumes of data,” recalls Howard Samm. “We were attracted to the SUSE solution as it provides greater flexibility and reliability, and we saw that it had huge potential to help us overcome the performance challenges we faced.
“In addition to implementing SUSE solutions, we also added another layer above our existing estate to help manage the distribution and replication of data at the core of our backup operations.”
Experian’s move to SUSE Linux Enterprise Server provided the company with more time to explore available options for a long-term solution that would protect its existing storage hardware investment and align with the IT team’s budget for future investment.
Howard Samm continues: “We spent several weeks exploring all of the options open to us and during our search we came across SUSE’s new storage offering—SUSE Enterprise Storage. We had previously looked at an earlier incarnation of Ceph, and found it to be somewhat fragile. Knowing that SUSE was now willing to support the technology as part of a commercial offering gave us confidence. After all, we thought, if SUSE expects to turn a profit, they must be con dent that the solution is stable!”
SUSE Enterprise Storage, based on open source Ceph technology, brings together open software and commodity hardware to provide a highly scalable, reliable and cost-effective software-defined storage solution for block, object and file storage types.
“SUSE Enterprise Storage stood out from all of the other vendors we considered and offered everything that we needed,” explains Howard Samm. “As a software-defined storage solution, it enabled us to achieve the performance and availability we needed without requiring us to make large ongoing investments in specialized storage hardware.”
“We had been using SUSE Linux Enterprise Server as one of our core operating systems for a number of years before we decided to introduce SUSE Enterprise Storage into our infrastructure. Having used SUSE solutions with great results for some time, we have come to trust and respect the SUSE brand—primarily because its products enable access to the cost-efficient innovative open source arena and, in our experience, are flexible and reliable.
“As enterprise storage technology develops, more and more companies are moving to open source platforms. We have been looking to move in this direction for a number of years, both for the potential cost savings and for the ease with which we can tailor solutions to the specific needs of our business.
“We have had a truly positive experience working with SUSE, and have particular appreciation for the approach of their team. The SUSE package includes comprehensive support and assistance, so you’re never left alone to trawl through support manuals or forums for advice when optimizing your environment or troubleshooting. This is especially reassuring when you are housing all your core business on SUSE products, because you know that SUSE will be with you every step of the way.
“The great support that SUSE provide is one of the main reasons I would recommend SUSE solutions to other businesses looking to diversify their architecture and capabilities. The open source world certainly offers significant benefits, but we think it’s best to get the technology backed by a supportive enterprise partner.”
Using both SUSE Linux Enterprise Server and SUSE Enterprise Storage, Experian has modernized its IT infrastructure and harnessed the benefits of software-defined storage.
Howard Samm says: “Moving to software-defined storage was an easy decision for us, and SUSE solutions presented the best point of entry for this type of architecture. We saw that switching to a software-defined approach would eliminate the need to manually allocate stores of data to specific hardware and enable us to leverage all the storage devices on our network to maximize availability.”
As the first stage in its use of SUSE Enterprise Storage, Experian migrated the vast majority of its VMware images to the solution. The company is now working to migrate the storage from its Microsoft Windows landscape, including a large remote desktop farm. Subsequently, Experian will tackle the larger and potentially riskier task of migrating its remaining NAS stores to SUSE Enterprise Storage. (In general, the company will seek to fully amortize its past technology investments, migrating NAS devices as and when they reach end of life). Finally, providing performance testing is successful, Experian will migrate its main production databases over from flash storage.
At the time of writing, Experian has six live nodes in its SUSE Enterprise Storage cluster, and is on the point of adding a further two nodes in a second location as a disaster-recovery option. The cluster currently holds around 115 TB of data, out of a total (and fast growing) capacity of approximately 1 PB across Experian.
READY FOR SOFTWARE-DEFINED SUCCESS
With SUSE Linux Enterprise Server and SUSE Enterprise Storage at the heart of its IT infrastructure, Experian can overcome the technical challenges posed by high data volumes and is prepared to meet the evolving demands of its clients.
Howard Samm says: “With SUSE Linux Enterprise Server and SUSE Enterprise Storage we have seen a dramatic increase in performance. With this high level of performance, we can eliminate latency during replication runs and, therefore, significantly reduce the risk of data loss or business disruption.
“The SUSE solution has also enabled us to increase our capacity and availability. Taking a software-defined approach to our storage environment not only enables us to manage our IT infrastructure more efficiently, it also maximizes the availability of our systems. With the added level of abstraction provided by this approach, we can automatically transfer workloads between devices—cutting out the cost and effort of manually migrating them. In this way, we eliminate the concern that individual servers could hit their capacity limits and cause issues for the network as a whole.”
He adds: “The stability of SUSE Enterprise Storage is a massive benefit—the cluster is extremely stable, and has proven itself to be highly tolerant of the failure of individual disks. Very little maintenance is required to keep it running smoothly, and we can patch in a completely transparent and non-disruptive way. The reduced administrative burden for storage maintenance means that we’re getting back up to one man-day per week.”
By unleashing the power of software-defined storage, Experian has simplified capacity planning and budgeting. Howard Samm continues: “With SUSE Linux Enterprise Server and SUSE Enterprise Storage we are able to get a much clearer picture of the performance of each device in our network as the amount of data we process increases. This greater level of clarity makes capacity planning for the future much easier.
“With easier capacity planning enabled by our software-defined approach to storage, we can ensure that we make the most cost-efficient use of storage in our architecture by sweating the assets we already have—reducing the need to make large capital investments on hardware that could ultimately lie redundant much of the time.”
Experian is now better equipped to provide rapid insights for the businesses it serves and has an enterprise storage platform that it can develop and innovate with ease.
Howard Samm concludes: “Our SUSE solutions have given us a foundational platform for seamless capacity and performance growth, with a clear path for future development.
We are now much better equipped to scale to meet peaks in demand and ensure a consistent, high-quality service for each and every client.”