CSI topology for FlashArray Direct Access volumes and FlashBlade Direct Access filesystems
The CSI topology feature for FlashArray Direct Access volumes and FlashBlade Direct Access filesystems allows applications to provision storage on a FlashArray Direct Access volume or FlashBlade Direct Access filesystem that is in the same set of Kubernetes nodes where the application pod is located.
Prerequisites
In order to use the CSI topology feature with a FlashArray Direct Access volume or FlashBlade Direct Access filesystem, you must meet the following prerequisites:
- Install Portworx version 2.11.0 or newer
- Install the Portworx Operator version 1.8.1 or newer
- Install Stork version 2.11 or newer
Enable CSI topology
When you enable CSI topology, you’ll specify Labels
that describe the topology for each FlashArray. The keys must match a set of specific strings, but you can define your own values. The following CSI topology Labels
keys are available:
topology.portworx.io/region
topology.portworx.io/zone
topology.portworx.io/datacenter
topology.portworx.io/provider
topology.portworx.io/row
topology.portworx.io/rack
topology.portworx.io/chassis
topology.portworx.io/node
topology.portworx.io/hypervisor
topology.purestorage.com
labels when you migrate from PSO to Portworx using the pso2px tool.
Enable on a new cluster
To enable the CSI topology feature, perform the following steps:
Add the following to your StorageCluster
spec
:csi: enabled: true topology: enabled: true
Create a
px-pure-secret
containing the information for your FlashArrays. IncludeLabels
that specify the topology for each FlashArray. The keys must match a set of specific strings, but you can define your own values. For example:{ "FlashArrays": [ { "MgmtEndPoint": "<managementEndpoint>", "APIToken": "<apiToken>", "Labels": { "topology.portworx.io/zone": "zone-0", "topology.portworx.io/region": "region-0" } } }
Label your Kubernetes nodes with labels that correspond to the
Labels
from the previous step. For example:kubectl label node <nodeName> topology.portworx.io/zone=zone-0 kubectl label node <nodeName> topology.portworx.io/region=region-0
Specify the placement strategy by defining the
nodeAffinity
in your Pod or StatefulSet. For example:spec: affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - key: topology.portworx.io/zone operator: In values: - zone-0 - key: topology.portworx.io/region operator: In values: - region-0
In your StorageClass, choose one of the following strategies so that the PVC uses your topology strategy:
Create a StorageClass with
volumeBindingMode
set toWaitForFirstConsumer
. For example:kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: fio-sc-fada provisioner: pxd.portworx.com parameters: backend: "pure_block" max_bandwidth: "10G" max_iops: "10G" csi.storage.k8s.io/fstype: ext4 volumeBindingMode: WaitForFirstConsumer allowVolumeExpansion: true
Create a StorageClass that defines
allowedTopologies
. For example:kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: fio-sc-fada provisioner: pxd.portworx.com parameters: backend: "pure_block" max_bandwidth: "10G" max_iops: "10G" csi.storage.k8s.io/fstype: ext4 allowedTopologies: - matchLabelExpressions: - key: topology.portworx.io/rack values: - rack-0 - rack-1
Enable on an existing cluster
Edit the cluster’s StorageCluster
spec
to include the following:csi: enabled: true topology: enabled: true
Edit the
px-pure-secret
for your FlashArray to include topologyLabels
using the following command:kubectl edit secrets px-pure-secret
Include
Labels
that specify the topology for each FlashArray. The keys must match a set of specific strings, but you can define your own values. For example:{ "FlashArrays": [ { "MgmtEndPoint": "<managementEndpoint>", "APIToken": "<apiToken>", "Labels": { "topology.portworx.io/zone": "zone-0", "topology.portworx.io/region": "region-0" } } }
Get all Portworx pods using the following command:
kubectl get pods -n kube-system -l name=portworx -o wide
Delete Portworx pods for each node one by one using the following command:
kubectl delete pods -n kube-system <px-pod-name>
Wait for the Portworx pods to come up in the node. You can monitor the pods after deletion using the following command:
kubectl get pods -n kube-system -l name=portworx -o wide | grep <node-name>
Delete the Portworx pods for next node. Repeat until Portworx pods are restarted for all nodes.
Wait for Portworx pods to be up in all nodes.
Validate that topology is enabled in a node by describing
csinode
with the following command:kubectl describe csinode <node-name>
Name: <node-name> ... Spec: Drivers: pxd.portworx.com: Node ID: <node-id> Topology Keys: [topology.portworx.io/region topology.portworx.io/zone]
Related topics
- Configure Pure Storage FlashArray as a Direct Access volume
- Configure Pure Storage FlashBlade as a Direct Access filesystem