[Kubernetes] AKS Premium Storage Persistent Volume 기능 검증

Block Blob

1. Premium_GRS Block Blob Storage Class를 생성합니다.

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: premiumblob-sc
provisioner: blob.csi.azure.com
parameters:
  resourceGroup: <ResourceGroupName>
  storageAccount: <StorageAccountName>
  server: <StorageAccountName>.blob.core.windows.net
  skuName: Premium_ZRS  # available values: Standard_LRS, Premium_LRS, Standard_GRS, Standard_RAGRS, Standard_ZRS, Premium_ZRS
reclaimPolicy: Delete
volumeBindingMode: Immediate
allowVolumeExpansion: true
mountOptions:
  - -o allow_other
  - --file-cache-timeout-in-seconds=120
  - --use-attr-cache=true
  - --cancel-list-on-mount-seconds=10  # prevent billing charges on mounting
  - -o attr_timeout=120
  - -o entry_timeout=120
  - -o negative_timeout=120
  - --log-level=LOG_WARNING  # LOG_WARNING, LOG_INFO, LOG_DEBUG
  - --cache-size-mb=1000  # Default will be 80% of available memory, eviction will happen beyond that.

 

2. PVC를 생성합니다.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: premiumblob-pvc
spec:
  accessModes:
  - ReadWriteMany
  storageClassName: premiumblob-sc
  resources:
    requests:
      storage: 5Gi

 

3. Deployment를 생성합니다.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: azureblob-premium-deploy
  labels:
    app: azureblob-premium-deploy
spec:
  replicas: 2
  selector:
    matchLabels:
      app: azureblob-premium-deploy
  template:
    metadata:
      labels:
        app: azureblob-premium-deploy
    spec:
      containers:
      - name: azureblob-premium-deploy
        image: mcr.microsoft.com/oss/nginx/nginx:1.17.3-alpine
        ports:
        - containerPort: 80
        volumeMounts:
          - mountPath: /mnt/blob
            name: volume
      volumes:
        - name: volume
          persistentVolumeClaim:
            claimName: premiumblob-pvc

 

 

Azure Blob NFS Protocol

1. Storage Account를 Data Lake Gen2 upgrade로 업그레이드 합니다.

2. Upgrade와 동시에 NFS v3 Protocol을 사용하도록 Enable 합니다.

3. NFS Protocol을 사용하는 Storage Class를 사용합니다.

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: azureblob-nfs
provisioner: blob.csi.azure.com
parameters:
  resourceGroup: <ResourceGroupName>
  storageAccount: <StorageAccountName>
  server: <StorageAccountName>.blob.core.windows.net
  protocol: nfs
  skuName: Premium_ZRS
volumeBindingMode: Immediate

 

4. Persistence Volume Claim을 생성합니다.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: azureblob-nfs-pvc
spec:
  accessModes:
  - ReadWriteMany
  storageClassName: azureblob-nfs
  resources:
    requests:
      storage: 5Gi

 

5. Deployment를 배포합니다.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: azureblob-nfs-deploy
  labels:
    app: azureblob-nfs-deploy
spec:
  replicas: 1
  selector:
    matchLabels:
      app: azureblob-nfs-deploy
  template:
    metadata:
      labels:
        app: azureblob-nfs-deploy
    spec:
      containers:
      - name: azureblob-nfs-deploy
        image: mcr.microsoft.com/oss/nginx/nginx:1.17.3-alpine
        ports:
        - containerPort: 80
        volumeMounts:
          - mountPath: /mnt/blob
            name: volume
      volumes:
        - name: volume
          persistentVolumeClaim:
            claimName: azureblob-nfs-pvc

 

 

Azure Blob Storage

 

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: <Name>
provisioner: blob.csi.azure.com
parameters:
  resourceGroup: <ResourceGroupName>
  storageAccount: <StorageAccountName>
  server: <StorageAccountName>.blob.core.windows.net
  skuName: Standard_ZRS  # available values: Standard_LRS, Premium_LRS, Standard_GRS, Standard_RAGRS, Standard_ZRS, Premium_ZRS
reclaimPolicy: Retain # Delete이 아닌 Retain으로 설정하는 경우 AKS 내부에서 PVC를 삭제하여도 Storage Account에는 PVC 데이터가 남아 있음(대신 다시 붙일 수 없음)
volumeBindingMode: Immediate
allowVolumeExpansion: true
mountOptions: # -o allow_other 옵션을 빼면 777에서 755
  - --file-cache-timeout-in-seconds=120
  - --use-attr-cache=true
  - --cancel-list-on-mount-seconds=10  # prevent billing charges on mounting
  - -o attr_timeout=120
  - -o entry_timeout=120
  - -o negative_timeout=120
  - --log-level=LOG_WARNING  # LOG_WARNING, LOG_INFO, LOG_DEBUG
  - --cache-size-mb=1000  # Default will be 80% of available memory, eviction will happen beyond that.
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: azureblob-zrs-pvc
spec:
  accessModes:
  - ReadWriteMany
  storageClassName: azureblob-zrs-sc
  resources:
    requests:
      storage: 5Gi

 

mountOptions에 -o allow_other이 들어가 있으면 777 없으면 755 입니다.

 

Blob은 HTTP/S 통신이기 때문에 Permission이 있을 수 없습니다.

하지만 아래와 같이 Permission이 확인되는 것은 Blob에서 Temp Directory로 내릴 때 Permission 입니다.

 

 

Azure Files

1. Azure Files Premium_ZRS Storage Class를 생성합니다.

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: azurefile-premium-sc
provisioner: file.csi.azure.com # replace with "kubernetes.io/azure-file" if aks version is less than 1.21
allowVolumeExpansion: true
parameters:
  resourceGroup: <ResourceGroupName>
  storageAccount: <StorageAccountName>
  server: <StorageAccountName>.blob.core.windows.net
  skuName: Premium_ZRS
reclaimPolicy: Delete
volumeBindingMode: Immediate
mountOptions:
  - dir_mode=0777
  - file_mode=0777
  - uid=0 #0472
  - gid=0 #0472
  - mfsymlinks
  - cache=strict
  - actimeo=30

 

2. PVC를 생성합니다.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: azurefile-premium-pvc
spec:
  accessModes:
    - ReadWriteMany
  storageClassName: azurefile-premium-sc
  resources:
    requests:
      storage: 10Gi

 

3. Deployment를 생성합니다.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: azurefile-premium-deploy
  labels:
    app: azurefile-premium-deploy
spec:
  replicas: 1
  selector:
    matchLabels:
      app: azurefile-premium-deploy
  template:
    metadata:
      labels:
        app: azurefile-premium-deploy
    spec:
      containers:
      - name: azurefile-premium-deploy
        image: mcr.microsoft.com/oss/nginx/nginx:1.15.5-alpine
        ports:
        - containerPort: 80
        volumeMounts:
          - mountPath: /mnt/azure
            name: volume
      volumes:
        - name: volume
          persistentVolumeClaim:
            claimName: azurefile-premium-pvc

 Premium Fileshare는 최소 PVC용량과 상관없이 최소 Provisioned Capa가 100GiB 입니다.