Install Portworx on PKS using the Operator
Before installing Portworx, let’s ensure the PKS environment is prepared correctly.
Ensure that your PKS cluster meets the minimum requirements for Portworx.
Enable privileged containers and kubectl exec
Ensure that the following options are selected on all plans on the PKS tile:
- Enable Privileged Containers
- Disable DenyEscalatingExec (this is useful for using
kubectl execto run
Enable zero downtime upgrades for Portworx PKS clusters
Use the following steps to add a runtime addon to the Bosh Director to stop the Portworx service.
Perform these steps on any machine where you have the bosh CLI.
Create and upload the release.
Replace director-environment below with the environment which points to the Bosh Director.
git clone https://github.com/portworx/portworx-stop-bosh-release.git cd portworx-stop-bosh-release mkdir src bosh create-release --final --version=1.0.0 bosh -e director-environment upload-release
Add the addon to the Bosh Director.
First, let’s fetch your current Bosh Director runtime config:
bosh -e director-environment runtime-config
If this is empty, you can simply use the runtime config at runtime-configs/director-runtime-config.yaml.
If you already have an existing runtime config, add the release and addon in runtime-configs/director-runtime-config.yaml to your existing runtime config.
Once we have the runtime config file prepared, let’s update it in the Director:
bosh -e director-environment update-runtime-config runtime-configs/director-runtime-config.yaml
Apply the changes.
After the runtime config is updated, go to your Operations Manager Installation Dashboard and click “Apply Changes”. This will ensure bosh will add the addon on all new vm instances.
If you already have an existing Portworx cluster, you will need to recreate the VM instances using the bosh recreate command.
For on-premises clusters, PKS (Pivotal Container Service) supports VMware vSphere.
The following diagram gives an overview of the Portworx architecture on vSphere using shared datastores.
- Portworx runs on each Kubernetes minion/worker.
- Based on the given spec by the end user, Portworx on each node will create its disk on the configured shared datastores or datastore clusters.
- Portworx will aggregate all of the disks and form a single storage cluster. End users can carve PVCs (Persistent Volume Claims), PVs (Persistent Volumes) and Snapshots from this storage cluster.
- Portworx tracks and manages the disks that it creates. In a failure event, if a new VM spins up, Portworx on the new VM will be able to attach to the same disk that was previously created by the node on the failed VM.
Install the Operator
Enter the following
kubectl create command to deploy the Operator:
kubectl create -f https://install.portworx.com/?comp=pxoperator
ESXi datastore preparation
Create one or more shared datastore(s) or datastore cluster(s) which is dedicated for Portworx storage. Use a common prefix for the names of the datastores or datastore cluster(s). We will be giving this prefix during Portworx installation later in this guide.
Step 1: vCenter user for Portworx
You will need to provide Portworx with a vCenter server user that will need to either have the full Admin role or, for increased security, a custom-created role with the following minimum vSphere privileges:
- Allocate space
- Browse datastore
- Low level file operations
- Remove file
- Local operations
- Reconfigure virtual machine
- Virtual machine
- Change Configuration
- Add existing disk
- Add new disk
- Add or remove device
- Advanced configuration
- Change Settings
- Extend virtual disk
- Modify device settings
- Remove disk
If you create a custom role as above, make sure to select “Propagate to children” when assigning the user to the role.
Step 2: Create a Kubernetes secret with your vCenter user and password
Update the following items in the Secret template below to match your environment:
- VSPHERE_USER: Use output of
echo '<vcenter-server-user>' | base64
VSPHERE_PASSWORD: Use output of
echo '<vcenter-server-password>' | base64
apiVersion: v1 kind: Secret metadata: name: px-vsphere-secret namespace: kube-system type: Opaque data: VSPHERE_USER: YWRtaW5pc3RyYXRvckB2c3BoZXJlLmxvY2Fs VSPHERE_PASSWORD: cHgxLjMuMEZUVw==
kubectl apply the above spec after you update the above template with your user and password.
Step 3: Generate the specs
Generate the spec file using the Portworx Spec Generator with the following configurations:
Under the Basic tab, ensure that the following config parameters are set.
- Select the Use the Portworx operator checkbox.
- Select the 2.11 version of Portworx in Portworx Version drop-down.
- Under ETCD, select Built-in.
Under the Storage tab, ensure that the following config parameters are set.
- Under Select your environment, choose Cloud.
- Under Select Cloud platform, select vSphere.
- Under Configure storage devices, choose Create Using a Spec in Select type of disk.
- Under vCenter Endpoint, type your vCenter IP or hostname.
- Under vCenter datastore prefix, type the prefix of the datastore for Portworx to use.
Under the Customize tab, ensure that the following config parameters are set.
- In Are you running either of these?, choose PKS (Pivotal Container Service).
- In Advanced settings, select the following checkboxes.
- Enable Stork
- Enable CSI
- Enable Monitoring
- Enable Telemetry
Apply the specs
Apply the generated specs to your cluster.
kubectl apply -f px-spec.yaml
Monitor the Portworx nodes
Enter the following
kubectl getcommand and wait until all Portworx nodes show as ready in the output:
kubectl -n kube-system get storagenodes -l name=portworx
Enter the following
kubectl describecommand with the name of one of the Portworx nodes to show the current installation status for individual nodes:
kubectl -n kube-system describe storagenode <portworx-node-name>
Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal PortworxMonitorImagePullInPrgress 7m48s portworx, k8s-node-2 Portworx image portworx/px-enterprise:2.5.0 pull and extraction in progress Warning NodeStateChange 5m26s portworx, k8s-node-2 Node is not in quorum. Waiting to connect to peer nodes on port 9002. Normal NodeStartSuccess 5m7s portworx, k8s-node-2 PX is ready on this nodeNOTE: In your output, the image pulled will differ based on your chosen Portworx license type and version.