Go back a year or two, and Portworx was one of a handful of companies that were laser focused on solving the issues for organizations looking to use containers with stateful applications. Containers and there easy creation and destruction are good and all, but if you have an application that relies on data being persistent, there is a bit of a disconnect with the container notion. Which is where Portworx came in. With Portworx, users can manage any database or stateful service on any infrastructure.

Sounds fair enough, right? Well, indeed but like any other company, Portworx has to ensure that it’s keeping up with current trends. And so, over the last year or two, Portworx has re-positioned from being a Docker-centric company to one that is more generally container focused and, as anyone who follows the space will know, that means having a good Kubernetes story to tell. As such, Portworx is now articulating a stateful proposition for not only Docker’s Swarm orchestration tooling, but also Mesosphere DC/OS and, perhaps most importantly, Kubernetes.

Across the various platforms it works with, Portworx is focused on solving what it suggests are the five most common problems DevOps teams encounter when running stateful services in production: persistence, high availability, data automation, security, and support for multiple data stores and infrastructure.

And so, news this week that Portworx is releasing an open source Kubernetes scheduler extender that is designed to layer additional capabilities on top of stateful applications being run on Kubernetes. STORK (nattily short for Storage Orchestrator Runtime for Kubernetes) is a project that takes advantage of a plugin interface that will allow it to be extended beyond just Portworx, but onto any storage driver for Kubernetes. Portworx is welcoming contributions to the STORK project from the general Kubernetes community. STORK provides hyperconvergence, failure-domain awareness, storage health monitoring and snapshot-lifecycle features.

Portworx handily provided a list of the key problems running stateful applications on Kubernetes and, by extension, how STORK resolves those issues

Maintaining hyperconvergence during failover

Modern stateful applications like Cassandra, Kafka, ElasticSearch and more scale out to increase capacity and perform best when each instance runs in close proximity to its data. Having local direct storage access reduces latency and improves the response times of these databases. For this reason, it is ideal to schedule a pod on a host where its data is located.  Today, DevOps teams have to use techniques such as labels, constraints or affinity/anti-affinity to ensure data locality. These rules are hard to manage when running applications at scale across a large number of servers and data centers, increasing room for error.

Kubernetes was designed with extenders in mind. STORK enables the deployment of complex distributed stateful applications by implementing a Kubernetes scheduler extender that influences pod scheduling based on the location of volumes that a pod requires. So even after failure events, pods are scheduled on hosts that already have a copy of data, ensuring ongoing hyperconvergence.

Failure domain awareness

Multiple-node distributed databases need to be deployed in a manner that recognizes the unique network topology of a customer’s cloud or data center. For instance, it is important not to schedule two instances of Cassandra on the same node or rack lest a failure of one server or rack bring down the application. Similar to the challenges of managing hyperconvergence, DevOps teams today need to use labels, constraints or affinity/anti-affinity rules to manage scheduling across failure domains.

Using the same scheduler extender pattern described above, STORK automatically manages anti-affinity to enforce scheduling of pods across failure domains.

Health monitoring of storage plugins

A common issue with stateful applications is the wear and tear induced by the storage fabric. The overall health of a pod will be subject to this wear and tear over time. This can result in pods not being able to be rescheduled to healthy hosts if a storage driver experiences a failure, resulting in application unavailability.

STORK solves this issue by failing over pods when the storage driver on a node goes into an error or unavailable state. This allows applications to be truly highly available without any user intervention.

Volume Snapshot support

Stateful applications have a complex life cycle that needs to be managed. Snapshots provide a key management tool for data recovery or duplication of an environment for testing. Today, DevOps teams have to manage these lifecycle operations using the tools provided by the storage provider instead of directly through Kubernetes, reducing the automation benefits of the Kubernetes platform for complex data workflows like DR, testing, upgrades, and blue-green deployments.

STORK adds support for orchestrating volume snapshots through Kubernetes. This allows users to take snapshots of PVCs and then restore those snapshots to other PVCs all through Kubernetes. This allows users to automate complex data workflows, all through Kubernetes.

MyPOV

The Kubernetes world, and the Docker world before it, are full of vendors trying to gain momentum and introducing open source initiatives as a way of doing so. The difficulty that all of these vendors have is to define a large-enough value proposition, and segment only a part of that to be delivered by the open source project. In doing so, they leave some value on the table, just waiting to be monetized by themselves.

It is, as might be expected, a complex job, and there are far more vendors who have been unsuccessful at this approach than those who have worked it out. It is also worth noting that CNCF, the foundation that manages Kubernetes, is quickly adding additional open source projects to its roster – key for Portworx will be to ensure CNCF anoints STORK in this way, such that it doesn’t get left out in the cold. In commenting about CNCF’s recent adoption of the Rook project as one of its initiatives, Michael Ferranti from Portworx scouted around this topic, as he said:

We’re excited to see the CNCF acknowledge the central importance of storage to the Kubernetes ecosystem.  We’ve heard loud and clear from our Fortune 100 customers that solving the storage problem is a key driver of their container adoption.  We see value in Rook as an operator to facilitate storage for Kubernetes.  We’re looking forward to Rook expanding its support for multiple storage backends.  There is a lot of interesting work going on right now in the community and we’re looking forward to CNCF adopting other interesting storage projects as part of the CNCF landscape.

Time will tell if STORK is the one – it’s a fair bet that Portworx certainly hopes so.

Ben Kepes

Ben Kepes is a technology evangelist, an investor, a commentator and a business adviser. Ben covers the convergence of technology, mobile, ubiquity and agility, all enabled by the Cloud. His areas of interest extend to enterprise software, software integration, financial/accounting software, platforms and infrastructure as well as articulating technology simply for everyday users.

Leave a Reply