In-Memory Database: What It Is, How It Works, and Best Use Cases

Powering real-time analytics at speed.

Benefits of In-Memory Databases:

  • Accelerates real-time analytics.

  • Decreases data latency.

  • Increases scalability for large-scale analytics.

author-image

Oleh

In-memory databases are helping to solve some of the biggest analytical problems in business today. As companies process massive amounts of data, real-time analytics and artificial intelligence applications require immediate access to that data.

Traditional storage media like Flash storage (SSD) and spinning hard-disk drives (HDD) haven’t been able to fill that need, while DRAM remains cost prohibitive and capacity constrained for many organizations.

61% of data management professionals say system constraints are their top concern when trying to get value from their data.1

What Is an In-Memory Database?

The data in a traditional database is stored on disk. When the data is needed, it’s called into the local system memory—or RAM—and then is processed by the CPU. Because it takes time to search for data that resides on disks, bottlenecks frequently occur.

In contrast, an in-memory database stores data directly in system memory, plugging directly into the high-speed, low-latency memory bus. It decreases data latency by reducing the time needed to analyze data. With an in-memory database, data is accessed much quicker, allowing you to analyze large volumes of complex data in real time.

70% of firms are implementing, have implemented, or are expanding adoption of an in-memory database.1

In-Memory vs. Traditional Database

To better understand the benefits of an in-memory database, or in-memory db, let’s take a deeper dive into how the different types of memory work. Until recently, data centers have been limited to the following memory and storage solutions:

  • DRAM memory is very fast, but has limited capacity and can be prohibitively expensive.
  • SSDs offer more capacity and lower cost than DRAM, but are orders of magnitude slower than DRAM.
  • HDDs can store huge amounts of data at lower prices but are even slower than SSDs.

As you can see, each of these options has some kind of limitation, whether in performance, capacity, or cost.

Persistent memory offers a new option for in-memory databases. It delivers performance close to DRAM, at larger capacities and more-affordable price points.2 Intel® Optane™ persistent memory provides an in-memory database with the best of both worlds: increased capacity over DRAM while avoiding the performance constraints of traditional storage media.

In-memory analytics helps improve software performance in other ways. It uses multicore, multithread processor capabilities, which can operate on data in real time due to the reduced latency of keeping data in memory.

In-Memory Database Use Cases

From finance and retail to healthcare and cybersecurity, persistent memory can help facilitate the real-time analytics you use to make the decisions that keep your business moving forward. In fact, real-time analytics depends on in-memory databases to deliver insights at the speeds required for these applications. The following use cases are based on a reference architecture built on Lenovo ThinkSystem Servers powered by Intel® Xeon® Scalable processors using on-disk memory.3 ThinkSystem Servers configured with 2nd Generation Intel® Xeon® Scalable processors support Intel® Optane™ persistent memory, which provides additional capacity, more affordability, and data persistence.4

Finance: Fraud Detection and Monitoring

Financial institutions can use real-time streaming analytics to improve business health and uncover insights for better decision-making. Machine learning models can be trained using previous transactional data to monitor credit transactions and predict fraudulent transactions.

Retail: Operations Efficiencies and Higher Sales Volumes

Retailers can use real-time streaming analytics to improve operations, track inventory, boost sales volumes, gather trend insights, and improve customer satisfaction.

Healthcare: Clinical Risk and Patient Safety Monitoring

Analytics in healthcare can help facilitate the use of big data and machine learning to improve organizational efficiency and provide better patient experiences. Administrators can use real-time analytics to assess clinical risk, monitor patient safety, personalize patient results, and reduce patient readmission.

Network Threat Detection

Machine learning models can perform streaming analysis of DNS data, proxies, and traffic monitoring to expedite threat detection.

The Importance of Persistent Memory

One of the big challenges with DRAM is the loss of data associated with power failures or shutdowns. Persistent memory reduces downtime cycles that occur when your server is powered off. Typically, when a machine is powered down, all of the running applications and data have to be retrieved again once the machine powers back up. Because the data in persistent memory is preserved between power cycles, an in-memory database is able to restart up to 13x faster than reloading data from disk.5

According to a Forrester survey of data management professionals, in-memory platforms with persistent memory technology:

  • Improve their ability to support real-time analytics (61%)
  • Reduce system downtime cycles (61%)
  • Increase scalability for large-scale analytical systems (58%)
  • Enable new AI, IoT, and real-time user experience (50%)6

Infrastructure Needs for In-Memory Databases

When building your in-memory database analytics stack, the first thing to consider is how you’ll use your data. This will determine the technology you should use to store and process your data. Consider your business needs. If they’re not time or memory intensive, you may opt for traditional solutions instead.

Here are some recommendations to follow when building your in-memory database infrastructure:

  • Put in place a distributed environment that gives you the flexibility to scale CPU, memory, and storage capacities as needed over time.
  • Create a relational, multimodal environment that supports SQL databases and semistructured data like JSON.
  • Ensure that data can be moved down from memory to storage tiers as it ages so you can optimize data management costs without compromising analytics or latency performance.

Intel® Technologies for In-Memory Databases

Intel’s range of data center solutions were designed to help simplify the process of using in-memory analytics by combining speed with high performance and capacity.

Intel® Xeon® Scalable Processors

Intel® Xeon® Scalable processors strengthen your most demanding applications by supporting in-memory analytics, artificial intelligence, high performance computing, and network transformation.

Intel® Optane™ Persistent Memory

Intel® Optane™ persistent memory combines the best of memory and storage traits to expand memory capacity while providing low-latency access to persistent data.

Intel® Optane™ Solid State Drive (SSD) for Data Center

The Intel® Optane™ Solid State Drive (SSD) for Data Center reduces bottlenecks in your data center storage and can speed up applications; allow bigger, more-affordable data sets than DRAM; and improve overall data center total cost of ownership (TCO). Unlike traditional NAND-based SSDs that have a “sweet spot” for high performance, our technology combines low latency with high random read/write performance.

Faster Insights, Better Performance

Companies are processing more data than ever before. But they also need quick access to insights as they work with memory-intensive analytics applications. For times when you need to process large amounts of data but need real-time insights, an in-memory database with persistent memory could be the option you’re looking for. By storing data in system memory, you can process more data faster, without worrying about downtime or sacrificing performance.

Speed time to insight while delivering engaging customer experiences with Intel® technology built for real-time analytics.