Create sharedv4 PVCs
This document describes how to use Portworx sharedv4 (ReadWriteMany) volumes in your Kubernetes cluster.
Prerequisites
- sharedv4 volumes must not be disabled on your cluster.
- NFS ports between nodes must be open.
Provision a Sharedv4 Volume
Sharedv4 volumes are useful when you want multiple PODs to access the same PVC (volume) at the same time. They can use the same volume even if they are running on different hosts. They provide a global namespace and the semantics are POSIX compliant.
To increase fault tolerance, you can enable Sharedv4 service volumes by setting a value for sharedv4_svc_type
. With this feature enabled, every sharedv4 volume has a Kubernetes service associated with it. Sharedv4 service volumes expose the volume via a Kubernetes service IP. If the sharedv4 (NFS) server goes offline for a sharedv4 service volume and the volume requires a failover, only application pods that were running on the 2 nodes involved in failover need to be restarted.
Notes about Sharedv4 service volumes:
- Sharedv4 service volumes are intended for use within the Kubernetes cluster where the volume resides.
- Sharedv4 service volumes default to using NFSv4.0.
- Known issues:
- On failover, Applications may receive an error for non idempotent requests. For example, if an
mkdir
call is issued prior to failover, the client can resend it to the new server, which returns anEEXIST
error if the directory was created by the first call.
- On failover, Applications may receive an error for non idempotent requests. For example, if an
Step 1: Create a StorageClass
Create the following storageClass, specifying your own values for the following fields:
- The
metadata.name
field with a name for your storageClass - The
parameters.repl
field with the replication factor you’d like to set - The
sharedv4
field set totrue
- (Optional) The
sharedv4_svc_type
field set toClusterIP
if you want to enable the sharedv4 service feature (Optional) The
sharedv4_failover_strategy
field set tonormal
oraggressive
(shorter failover grace period)NOTE: The default value forsharedv4_failover_strategy
in sharedv4 volumes isnormal
, and the default value forsharedv4_failover_strategy
in sharedv4 service volumes isaggressive
.kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: px-sharedv4-sc provisioner: kubernetes.io/portworx-volume parameters: repl: "2" sharedv4: "true" sharedv4_svc_type: "ClusterIP"
- The
Apply the storageClass:
kubectl apply -f examples/volumes/portworx/portworx-sharedv4-sc.yaml
Verify the storage class is created:
kubectl describe storageclass px-sharedv4-sc
Name: px-sharedv4-sc IsDefaultClass: No Annotations: <none> Provisioner: kubernetes.io/portworx-volume Parameters: repl=2,sharedv4=true,sharedv4_svc_type=ClusterIP Events: <none>
Step 2: Create persistent volume claim
Creating a ReadWriteMany persistent volume claim:
kubectl create -f examples/volumes/portworx/portworx-volume-sharedv4-pvc.yaml
Example:
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: px-sharedv4-pvc
annotations:
volume.beta.kubernetes.io/storage-class: px-sharedv4-sc
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 10Gi
Note the accessMode for this PVC is set to ReadWriteMany
so the Kubernetes allows mounting this PVC on multiple pods.
Verifying persistent volume claim is created:
kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESSMODES STORAGECLASS AGE
px-sharedv4-pvc Bound pvc-a38996b3-76e9-11e7-9d47-080027b25cdf 10Gi RWX px-sharedv4-sc 12m
Step 3: Create pods which use the persistent volume claim
We will start two pods which use the same shared volume.
Starting pod-1
kubectl create -f examples/volumes/portworx/portworx-volume-sharedv4-pod-1.yaml
Example:
apiVersion: v1
kind: Pod
metadata:
name: pod1
spec:
containers:
- name: test-container
image: gcr.io/google_containers/test-webserver
volumeMounts:
- name: test-volume
mountPath: /test-portworx-volume
volumes:
- name: test-volume
persistentVolumeClaim:
claimName: px-sharedv4-pvc
Starting pod-2
kubectl create -f examples/volumes/portworx/portworx-volume-sharedv4-pod-2.yaml
Example:
apiVersion: v1
kind: Pod
metadata:
name: pod2
spec:
containers:
- name: test-container
image: gcr.io/google_containers/test-webserver
volumeMounts:
- name: test-volume
mountPath: /test-portworx-volume
volumes:
- name: test-volume
persistentVolumeClaim:
claimName: px-sharedv4-pvc
Verifying pods are running:
kubectl get pods
NAME READY STATUS RESTARTS AGE
pod1 1/1 Running 0 2m
pod2 1/1 Running 0 1m
Convert a sharedv4 volume to a sharedv4 service volume
Perform the following steps to convert a sharedv4 volume to use the new sharedv4 service feature:
- Detach the volume by scaling down the application pods down to 0.
Run the following
pxctl
command:pxctl volume update --sharedv4_service_type=ClusterIP <volume>
Scale the pods back up to start the application.
Convert a sharedv4 service volume to a sharedv4 volume
Perform the following steps to convert a sharedv4 service volume to a sharedv4 volume:
- Detach the volume by scaling the application pods down to 0.
Run the following
pxctl
command:pxctl volume update --sharedv4_service_type="" <volume>
Scale the pods back up to start the application.
Update a legacy shared volume to a sharedv4 volume
You can update an existing shared volume to use the new v4 protocol and convert it into a sharedv4 volume. Run the following pxctl command to update the volume setting
pxctl volume update --sharedv4=on <vol-name>