Regular readers will be aware that I “dabble” from time to time in the world of Cloud Native Apps. For me, a lot of this dabbling is trying to figure out how I can go about providing persistent storage to container based applications. Typically this in the shape of container volumes that are carved out of the underlying storage infrastructure, whether that is VMFS, NFS, vSAN or even Virtual Volumes. VMware Project Hatchway has enabled me to do this on multiple occasions. Project Hatchway was officially announced at VMworld 2017, but I’ve been working with this team since the early days of the “docker volume driver for vSphere”. Project Hatchway now includes the vSphere Docker Volume Service (vDVS) plugin AND a vSphere Cloud Provider (vCP) to provide container volumes for Kubernetes. With Kubecon17 in Austin this week, there have been some exciting news on the Project Hatchway front that I want to share with you in this post.
The first piece of news is that Project Hatchway is now included in the Container Storage Interface (CSI), an initiative from the {code} team over at DELL Tech. This means that using CSI, you will not have to understand the intricacies of each of the different underlying storage systems in order to provision persistent container storage. CSI sits between the storage provider and the orchestration, and using a common set of commands, allows you to provision persistent container storage in a uniform manner each and every time, no matter the underlying storage infrastructure. There is a good write-up on CSI, as well as some details about the Project Hatchway (vSphere) inclusion.
The next piece of news comes from the Pivotal Container Service (PKS) initiative. This was also an announcement from VMworld 2017, which talks about how using Pivotal’s BOSH, we now have a very simple mechanism for deploying Kubernetes on vSphere. Project Hatchway’s VCP is now part of PKS, and its role is once again to provide persistent storage to containers that are instantiated and managed by Kubernetes. This integration includes the ability to select and set policies for container volumes that are deployed on storage such as VMware vSAN. Kubernetes supports a wide range of storage types, such as Persistent Volumes, Storage Classes and Stateful Sets. I’ve not delved in too deep into PKS yet, so I’m not 100% sure if it supports all of these different storage types. I don’t see why not, but something I need to research a bit more.
I mentioned already that Project Hatchway provides persistent storage for Kubernetes containers using VCP, the vSphere Cloud Provider component. I also previously said that this is now being used by PKS for container volumes. However, it is important to understand that you can still use VCP outside of PKS too, for example if you are running Kubernetes directly on top of vSphere. One of the main feature requests, according to Tushar Thole (engineering manager for Project Hatchway) is the ability to support Kubernetes clusters that span across multiple vCenters and datacenters. VCP now supports this.
The vSphere Docker Volume Service has been the stalwart of Project Hatchway, and has been around for at least 18 months now, with constant improvements all the time. One of those enhancements was changing vDVS to a docker plugin, and it also became one of the first docker plugins to support Windows as a Container Host/OS. I’ve used vDVS regularly to provide persistent storage for my containers, such as my “Project Harbor” registry and most recently for my testing of S3 object stores on top of vSAN. However Tushar informed me recently that the vDVS can now also be used with Virtual Volumes (VVols), which is very interesting and something I need to take a closer look at soon as well.
So as you can see, there is a lot of stuff going on around Project Hatchway. However it would be remiss of me not to mention vFile. vFile provides a way of doing shared file storage for cloud-native applications. This is still in its experimental phase, but it getting a lot of interest from those folks who have been putting it through its paces.
It’s great to see the traction that Project Hatchway is now getting. If this is something you are interested in getting started with, check it out on github here. And the engineering team are always looking for feedback, so feel free to leave feedback either here on this post or on github.
The post Project Hatchway hitting the mainstream – persistent storage for containers appeared first on CormacHogan.com.