Infra As Code — Terraform (5) Add New EBS Volume To AWS EKS Cluster

$ kubectl get storageclass
NAME PROVISIONER AGE
gp2 (default) kubernetes.io/aws-ebs 125m

Why Storage Volumes in K8s?

EBS vs. EFS

EFS vs EBS
  1. Grant proper permission to worker nodes.
  2. Install EBS CSI driver.
  3. Create a storage class and persistent volume claim.
  4. Add the storage claim to your pod.
  1. IAM Policy: If you followed my previous tutorial: “Infra As Code — Terraform (3) Add Worknodes To AWS EKS Cluster”. What you need to do is simple, add a new IAM policy and attach it to your EKS worknode:
Define new policy in "eks-nodes.tf"resource "aws_iam_policy" "eks_worknode_ebs_policy" {
name = "Amazon_EBS_CSI_Driver"

policy = <<POLICY
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:AttachVolume",
"ec2:CreateSnapshot",
"ec2:CreateTags",
"ec2:CreateVolume",
"ec2:DeleteSnapshot",
"ec2:DeleteTags",
"ec2:DeleteVolume",
"ec2:DescribeInstances",
"ec2:DescribeSnapshots",
"ec2:DescribeTags",
"ec2:DescribeVolumes",
"ec2:DetachVolume"
],
"Resource": "*"
}
]
}
POLICY
}
# And attach the new policy
resource "aws_iam_role_policy_attachment" "worknode-AmazonEBSCSIDriver" {
policy_arn = aws_iam_policy.eks_worknode_ebs_policy.arn
role = aws_iam_role.eks_worknode.name
}
$ kubectl apply -k "github.com/kubernetes-sigs/aws-ebs-csi-driver/deploy/kubernetes/overlays/stable/?ref=master"serviceaccount/ebs-csi-controller-sa created
clusterrole.rbac.authorization.k8s.io/ebs-external-attacher-role created
clusterrole.rbac.authorization.k8s.io/ebs-external-provisioner-role created
clusterrolebinding.rbac.authorization.k8s.io/ebs-csi-attacher-binding created
clusterrolebinding.rbac.authorization.k8s.io/ebs-csi-provisioner-binding created
deployment.apps/ebs-csi-controller created
daemonset.apps/ebs-csi-node created
csidriver.storage.k8s.io/ebs.csi.aws.com created
$ ls pod-specs/
claim.yaml storageclass.yaml
$ cat storage-specs/storageclass.yaml
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: ebs-sc
provisioner: ebs.csi.aws.com
parameters:
type: gp2
mountOptions:
- debug
volumeBindingMode: Immediate
$ cat pod-specs/claim.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: ebs-claim
spec:
accessModes:
- ReadWriteOnce
storageClassName: ebs-sc
resources:
requests:
storage: 4Gi
$ kubectl apply -f pod-specs/
persistentvolumeclaim/ebs-claim created
storageclass.storage.k8s.io/ebs-sc created
$ cat pod-specs/pod.yaml
apiVersion: v1
kind: Pod
metadata:
name: app
spec:
containers:
- name: app
image: centos
command: ["/bin/sh"]
args: ["-c", "while true; do echo $(date -u) >> /data/out.txt; sleep 5; done"]
volumeMounts:
- name: persistent-storage
mountPath: /data
volumes:
- name: persistent-storage
persistentVolumeClaim:
claimName: ebs-claim
$ kubectl apply -f pod-specs/pod.yaml
pod/app created
$ kubectl describe pod app
...
Volumes:
persistent-storage:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: ebs-claim
ReadOnly: false
default-token-mhq98:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-mhq98
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling 2m34s (x3 over 2m47s) default-scheduler pod has unbound immediate PersistentVolumeClaims
Normal Scheduled 2m29s default-scheduler Successfully assigned default/app to ip-10-0-0-11.ec2.internal
Normal SuccessfulAttachVolume 2m27s attachdetach-controller AttachVolume.Attach succeeded for volume "pvc-20ad39ee-f8dc-4b4a-92e2-cda09c073f4b"
Normal Pulling 2m24s kubelet, ip-10-0-0-11.ec2.internal Pulling image "centos"
Normal Pulled 2m19s kubelet, ip-10-0-0-11.ec2.internal Successfully pulled image "centos"
Normal Created 2m18s kubelet, ip-10-0-0-11.ec2.internal Created container app
Normal Started 2m18s kubelet, ip-10-0-0-11.ec2.internal Started container app

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store