Keeping State in a Stateless World: A 2025 Kubernetes Storage Showdown

Kubernetes Storage Showdown: Ceph Rook vs. Portworx vs. OpenEBS vs. Longhorn

Kubernetes is a modular platform that, by default, supports ephemeral storage—meaning data disappears when a pod is destroyed. To extend its functionality and manage persistent data, we use the Container Storage Interface (CSI). The CSI acts as a standard API that allows Kubernetes to connect to a wide array of storage systems.

The process is straightforward:

1. A developer defines their storage needs by creating a PersistentVolumeClaim (PVC).
2. The CSI driver, acting as an operator, listens for this claim.
3. The driver then automatically provisions a PersistentVolume (PV) that satisfies the claim's requirements.

For more information on creating PVCs and PVs, please read this article.

While the process is standardised, the key difference lies in the underlying storage provider, defined by a StorageClass. This choice, whether it's Ceph (managed by Rook), Portworx, Longhorn, or another solution, determines how your data is managed.

Why Do We Need Persistent, Remote Storage?

Unlike traditional virtual machine platforms where compute and storage often lived on the same machine, container orchestration aims for a more flexible "desired state." If we lose a node, we need the ability for workloads to quickly restart on another node and reconnect to their data seamlessly.

The core idea is that if a node fails, Kubernetes can reschedule the pods to a healthy node, which then attaches the same storage volume, ensuring data continuity and minimal downtime. This concept of remote, persistent storage serves several critical purposes:

• High Availability: Ensures data is always accessible, even during node failures.
• Resiliency: Provides data redundancy and protection against loss.
• Performance: Delivers the necessary IOPS (Input/Output Operations Per Second) to meet application demands.

A Practical Use Case: WordPress

Imagine you're running a high-availability WordPress site. This setup has two distinct data requirements:

1. The Database: A MariaDB StatefulSet needs block storage to store its structured, binary data.
2. Static Files: Unstructured data like images, themes, and plugins must be shared across all WordPress pods.

For the database, the common solution is ReadWriteOnce (RWO) storage. Each MariaDB instance gets its own dedicated block volume that cannot be attached to other pods simultaneously, ensuring data integrity.

For the static wp-content files, we need storage that can be accessed by multiple pods at the same time. This is achieved with ReadWriteMany (RWX) storage, which functions like a shared network drive. Because of the complexities of managing a distributed file system, RWX volumes typically have higher latency and lower performance than their RWO counterparts.

A more modern alternative for shared files is to use an S3-compatible object store. Instead of mounting a shared directory, the application uses an S3 API to store and retrieve files. This approach simplifies the architecture and can be implemented on-premises using a MinIO cluster backed by high-performance RWO block storage.

What are the Available CSI Options?

The Kubernetes storage ecosystem is vast, with options ranging from open-source community projects to powerful proprietary platforms. Choosing the right one requires carefully balancing your organisation's technical needs, operational expertise, and budget.

Below, we compare four of the most popular solutions: Ceph (via Rook), Portworx, Longhorn, and OpenEBS.

Feature Comparison

This table outlines the core features supported by each solution.

Strategic Comparison: Pros, Cons, and Use Cases

This table provides a high-level overview to help guide your decision based on your team's expertise, budget, and technical requirements.

Performance Comparison 📊

Finally, let's look at performance. The following benchmark results (sourced from an independent analysis ) highlight how these solutions stack up under various workloads. The top performer for each metric (Read and Write) is in bold.

Disclaimer: Performance benchmarks are highly dependent on the specific hardware, network infrastructure, and test configuration used. These results represent a single snapshot and may vary significantly in different environments.

Random IOPS

This chart shows the number of random read and write operations per second. Higher is better, as higher values indicate better performance for quickly accessing random data points, which is crucial for database workloads.

Bandwidth

This chart displays the throughput for sustained read and write operations in megabytes per second (MiB/s). Higher is better, as more bandwidth is ideal for applications that move large files.

Average Latency

This chart illustrates the time it takes to complete a single read or write operation, measured in microseconds (µs). For latency, lower is better, as it signifies a more responsive system with less waiting time for operations to complete.

Sequential Bandwidth

This chart shows the read and write throughput for large, sequential files, such as video streams or large backups. Higher is better, as it means large files can be read or written faster.

Mixed Random IOPS

This chart measures performance in a mixed workload scenario (typically 70% read, 30% write), simulating real-world application behavior. Higher is better, indicating a stronger ability to handle complex, simultaneous demands.