Elastic Kubernetes Service (EKS) using eksctl


This article provides instructions for installing Portworx on Elastic Kubernetes Service (EKS) using the Weaveworks eksctl command-line utility.

Prerequisites

Before you can install Portworx on EKS using pxctl, you must meet the following prerequisites:

  • You must have eksctl downloaded and installed on your local computer

Grant Portworx the needed AWS permissions

Portworx creates and attaches EBS volumes. As such, it needs the AWS permissions to do so. Below is a sample policy describing these permissions:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "<stmt-id>",
            "Effect": "Allow",
            "Action": [
                "ec2:AttachVolume",
                "ec2:ModifyVolume",
                "ec2:DetachVolume",
                "ec2:CreateTags",
                "ec2:CreateVolume",
                "ec2:DeleteTags",
                "ec2:DeleteVolume",
                "ec2:DescribeTags",
                "ec2:DescribeVolumeAttribute",
                "ec2:DescribeVolumesModifications",
                "ec2:DescribeVolumeStatus",
                "ec2:DescribeVolumes",
                "ec2:DescribeInstances",
                "autoscaling:DescribeAutoScalingGroups"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}

You can provide these permissions to Portworx in one of following ways:

  1. Instance Privileges: Provide above permissions for all the instances in the autoscaling cluster by applying the corresponding IAM role. More info about IAM roles and policies can be found here
  2. Environment Variables: Create a User with the above policy and provide the security credentials (AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY) to Portworx.

Create a ClusterConfig

The ClusterConfig dictates what resources eksctl requests from EKS for the purposes of running Portworx. Portworx requires a number of default resources and configurations in order to function, but other areas of your configuration will vary based on your needs.

  1. Create a ClusterConfig configuration YAML file, specifying your own configuration options for the following:

    • metadata:
      • name: withe cluster name you desire
      • region: with the region you want your eks service to operate from
      • version: with a supported EKS version
    • managedNodeGroups:
      • storage-nodes.instance: with the instance type appropriate for your workloads
      • storage-nodes.minSize: and storage-nodes.maxSize: with the number of worker nodes. Both values must be the same, and a minimum of 3.
      • storage-nodes.ssh.publicKeyPath: if no path is specified, the default will be id_rsa
      • storage-nodes.iam.attachPolicyARNs: with the ARN of the IAM policy you created for Portworx in the Grant Portworx the needed AWS permissions step
      • storageless-nodes.instanceType: with the instance type approprirate for your storageless node workloads
      • storageless-nodes.minSize: with the minimum number of storageless nodes that can be active on your cluster at any given time
      • storageless-nodes.maxSize: with the maximum number of storageless allowed on your cluster
      • storageless-nodes.desiredCapacity: with the ideal number of storageless nodes preferred on your cluster
      • storageless-nodes.iam.attachPolicyARNs: with the ARN of the IAM policy you created for Portworx in the Grant Portworx the needed AWS permissions step
    • availabilityZones: with the availabilty zones applicable to your region
    apiVersion: eksctl.io/v1alpha5
    kind: ClusterConfig
    metadata:
      name: px-eksctl
      region: us-east-1
      version: "1.14"
    managedNodeGroups:
      - name: storage-nodes
        instanceType: m4.xlarge
        minSize: 3
        maxSize: 3
        volumeSize: 20
        #ami: auto
        amiFamily: AmazonLinux2
        labels: {role: worker, "px/node-type": "storage"}
        tags:
          nodegroup-role: worker
        ssh:  
          allow: true
          publicKeyPath: ~/.ssh/aws-vm.pub
        iam:
          attachPolicyARNs:
            - arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy
            - arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy
            - arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly
            - arn:aws:iam::aws:policy/ElasticLoadBalancingFullAccess
            - <arn-of-your-portworx-aws-iam-policy>
          withAddonPolicies:
            imageBuilder: true
            autoScaler: true
            ebs: true
            fsx: true
            efs: true
            albIngress: true
            cloudWatch: true
      - name: storageless-nodes
        instanceType: m4.xlarge
        minSize: 3
        maxSize: 6
        desiredCapacity: 4
        volumeSize: 20
        amiFamily: AmazonLinux2
        labels: {role: worker}
        tags:
          nodegroup-role: worker-storageless
        ssh:
          allow: true
          publicKeyPath: ~/.ssh/aws-vm.pub
        iam:
          attachPolicyARNs:
            - arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy
            - arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy
            - arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly
            - arn:aws:iam::aws:policy/ElasticLoadBalancingFullAccess
            - <arn-of-your-portworx-aws-iam-policy>
          withAddonPolicies:
            imageBuilder: true
            autoScaler: true
            ebs: true
            fsx: true
            efs: true
            albIngress: true
            cloudWatch: true
    availabilityZones: [ 'us-east-1a', 'us-east-1b', 'us-east-1c' ]
  2. Enter the following eksctl create cluster command, specifying the name of the clusterConfig file you created in the step above:

    eksctl create cluster -f <my-clusterConfig>.yml

Generate the spec

To install Portworx with Kubernetes, you must first generate Kubernetes manifests that you will deploy in your cluster:

  1. Navigate to PX-Central and log in or create an account
  2. Select Install and Run to open the Spec Generator

    Screenshot showing install and run

  3. Select New Spec

    Screenshot showing new spec button

  4. Generate a spec with the following selections:

    • On the Storage tab, specify AWS and configure your storage devices based on your needs
    • On the Customize tab, select the Amazon Elastic Container Service for Kubernetes (EKS) option

Apply the specs

Apply the generated specs to your cluster.

kubectl apply -f px-spec.yaml
Monitor the Portworx pods

Wait till all Portworx pods show as ready in the below output:

kubectl get pods -o wide -n kube-system -l name=portworx
Monitor Portworx cluster status
PX_POD=$(kubectl get pods -l name=portworx -n kube-system -o jsonpath='{.items[0].metadata.name}')
kubectl exec $PX_POD -n kube-system -- /opt/pwx/bin/pxctl status

Post-Install

Once you have a running Portworx installation, below sections are useful.

Further reading

  • Refer to the eksctl github for more examples of config files which can be used as input to eksctl
  • For more information on what eksctl is, as well as how it works, refer to the eksctl documentation

Last edited: Thursday, Apr 16, 2020