您好,登錄后才能下訂單哦!
kubeadm k8s配置 ceph rbd存儲(chǔ) 類型storageClass
k8s的storageclass,實(shí)現(xiàn)pvc的動(dòng)態(tài)創(chuàng)建,綁定
首先安裝ceph參考https://blog.csdn.net/qq_42006894/article/details/88424199
1、部署rbd-provisioner
git clone https://github.com/kubernetes-incubator/external-storage.git
cd external-storage/ceph/rbd/deploy
NAMESPACE=ceph
sed -r -i "s/namespace: [^ ]+/namespace: $NAMESPACE/g" ./rbac/clusterrolebinding.yaml ./rbac/rolebinding.yaml
kubectl -n $NAMESPACE apply -f ./rbac
2、創(chuàng)建secret 和 pool
###創(chuàng)建admin secret
ceph auth get client.admin 2>&1 |grep "key = " |awk '{print $3'} |xargs echo -n > /tmp/key
kubectl create secret generic ceph-admin-secret --from-file=/tmp/key --namespace=ceph --type=kubernetes.io/rbd
###創(chuàng)建ceph osd pool
ceph osd pool create kube 128 128
ceph auth add client.kube mon 'allow r' osd 'allow rwx pool=kube'
###創(chuàng)建 用戶secret
ceph auth get-key client.kube > /tmp/key1
kubectl create secret generic ceph-secret --from-file=/tmp/key1 --namespace=ceph --type=kubernetes.io/rbd
3、創(chuàng)建storageclass
cat ceph-sc-ceph.yml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: ceph-rbd-ceph
namespace: ceph
annotations:
storageclass.kubernetes.io/is-default-class: "true"
provisioner: ceph.com/rbd
parameters:
monitors: 172.16.13.198:6789,172.16.13.199:6789,172.16.13.200:6789
adminId: admin
adminSecretName: ceph-admin-secret
adminSecretNamespace: ceph
pool: kubernetes
userId: kubernetes
userSecretName: ceph-secret
userSecretNamespace: ceph
fsType: ext4
imageFormat: "2"
imageFeatures: "layering"
4、配置rdb-provisioner的rbac權(quán)限控制
cd external-storage/ceph/rbd/deploy
修改里面的默認(rèn)命名空間就即可
cat clusterrolebinding.yaml
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: rbd-provisioner1
subjects:
roleRef:
kind: ClusterRole
name: rbd-provisioner1
apiGroup: rbac.authorization.k8s.io
cat clusterrole.yaml
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: rbd-provisioner1
namespace: ceph
rules:
apiGroups: [""]
resources: ["endpoints"]
verbs: ["get", "list", "watch", "create", "update", "patch"]
cat deployment.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: rbd-provisioner1
namespace: ceph
spec:
replicas: 1
strategy:
type: Recreate
template:
metadata:
labels:
app: rbd-provisioner1
spec:
containers:
cat rolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: rbd-provisioner1
namespace: ceph
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: rbd-provisioner1
subjects:
cat role.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: rbd-provisioner1
namespace: ceph
rules:
cat serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: rbd-provisioner1
namespace: ceph
kubectl apply -f .
5、創(chuàng)建pvc
cat ceph-pv-ceph.yml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: test-claim-ceph
namespace: ceph
spec:
accessModes:
kubectl apply -f ceph-pv-ceph.yml
查看,如果是bound 狀態(tài)說明成功
#kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
claim1 Bound pvc-ff3e450b-4629-11e9-9740-080027a073ff 1Gi RWO rbd 51m
如果出錯(cuò),定位錯(cuò)誤方法
kubectl describe pvc/claim1
kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
mysql-wayne-8744474d7-kz29c 1/1 Running 0 5d19h 10.244.6.85 node2 <none> <none>
rbd-provisioner-67b4857bcd-8vsb8 1/1 Running 0 63m 10.244.3.7 node04 <none> <none>
#查看到 rbd-provisioner 在 node04 上,然后再node04上查看容器log
docker logs -f 6ed00b76cb55 # 6ed00b76cb55 是容器的id (docker ps 查看)
6,遇到的坑
error retrieving resource lock default/ceph.com-rbd: endpoints “ceph.com-rbd” is forbidden: User “system:serviceaccount:default:rbd-provisioner” cannot get endpoints in the namespace “default”
在 rbd-provisioner.yaml 中 clusterrole最后添加
其他參考https://blog.csdn.net/qq_34857250/article/details/82562514
7、
參考:https://github.com/kubernetes-incubator/external-storage/tree/master/ceph/rbd
PS設(shè)置默認(rèn)storageclass:
kubectl patch storageclass ceph-rbd -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'
錯(cuò)誤提示:
MountVolume.WaitForAttach failed for volume "pvc-23555713-da9e-11e9-931b-000c29158017" : fail to check rbd image status with: (executable file not found in $PATH), rbd output: ()
解決辦法
在K8s各個(gè)節(jié)點(diǎn)執(zhí)行
yum install -y ceph-common
免責(zé)聲明:本站發(fā)布的內(nèi)容(圖片、視頻和文字)以原創(chuàng)、轉(zhuǎn)載和分享為主,文章觀點(diǎn)不代表本網(wǎng)站立場,如果涉及侵權(quán)請(qǐng)聯(lián)系站長郵箱:is@yisu.com進(jìn)行舉報(bào),并提供相關(guān)證據(jù),一經(jīng)查實(shí),將立刻刪除涉嫌侵權(quán)內(nèi)容。