溫馨提示×
您好，登錄后才能下訂單哦！
點(diǎn)擊 登錄注冊(cè) 即表示同意《億速云用戶服務(wù)條款》
您好，登錄后才能下訂單哦！
今年3月份在公司的內(nèi)部k8s培訓(xùn)會(huì)上，開發(fā)同事表示使用dashboard的可以滿足日常開發(fā)需求，例如查看pod的日志，執(zhí)行exec指令，查看pod的運(yùn)行狀態(tài)等，但對(duì)basic認(rèn)證的權(quán)限控制表示擔(dān)憂。
之前介紹過在1.5.2版本上部署dashboard服務(wù)，在1.9.1版本離線部署中，也介紹過dashboard服務(wù)的RBAC配置和使用技巧。因此本文將在前文基礎(chǔ)上完善Heapster的整合與利用token對(duì)用戶權(quán)限進(jìn)行控制。
dashboard的特點(diǎn)主要如下：
1、能夠直觀的看到rc、deployment、pod、services等k8s組件的運(yùn)行情況和日志信息。
2、結(jié)合heapster和influxdb后，dashboard的監(jiān)控圖表上可以看到pod的cpu和內(nèi)存消耗情況。
1、Heapster是容器集群監(jiān)控和性能分析工具，支持Kubernetes和CoreOS。
2、K8S集群的HPA功能的實(shí)現(xiàn)就依賴于這些metric數(shù)據(jù)，HPA將Heapster作為Resource Metrics API，向其獲取metric。
3、Kubernetes有個(gè)cAdvisor監(jiān)控（在1.9版本里面，cAdvisor已經(jīng)和kubelet整合在一起）。
在每個(gè)kubernetes
Node上都會(huì)運(yùn)行cAdvisor，它會(huì)收集本機(jī)以及容器的監(jiān)控?cái)?shù)據(jù)(cpu,memory,filesystem,network,uptime)。Heapster是一個(gè)收集者，Heapster可以收集Node節(jié)點(diǎn)上的cAdvisor數(shù)據(jù)，將每個(gè)Node上的cAdvisor的數(shù)據(jù)進(jìn)行匯總，還可以按照kubernetes的資源類型來集合資源，比如Pod、Namespace，可以分別獲取它們的CPU、內(nèi)存、網(wǎng)絡(luò)和磁盤的metric。默認(rèn)的metric數(shù)據(jù)聚合時(shí)間間隔是1分鐘。還可以把數(shù)據(jù)導(dǎo)入到第三方工具(如InfluxDB)。
2、Influxdb數(shù)據(jù)庫的相關(guān)知識(shí)介紹，可參考文檔：https://www.jianshu.com/p/d2935e99006e
2、如果對(duì)Heapster收集到的metric數(shù)據(jù)沒有持久化的需求，可以不配置Influxdb數(shù)據(jù)庫
3、本文Influxdb數(shù)據(jù)庫的存儲(chǔ)采用emptydir的方式實(shí)現(xiàn)，實(shí)際使用過程中，可以選擇吧Influxdb數(shù)據(jù)庫部署在k8s集群外部，或者使用其他存儲(chǔ)方案。
4、如果有需要的話，還可以集成一個(gè)grafana做web展示。Grafana配置可參考文檔：https://blog.51cto.com/ylw6006/2084403
需要科學(xué)上網(wǎng)方式獲取到dashboard相關(guān)的鏡像文件，倉庫可納入本地倉庫統(tǒng)一管理
# cat /etc/systemd/system/docker.service.d/http-proxy.conf [Service] Environment="HTTP_PROXY=http://192.168.115.2:1080"# systemctl daemon-reload# systemctl restart docker# docker pull k8s.gcr.io/kubernetes-dashboard-amd64:v1.8.3# docker pull k8s.gcr.io/heapster-influxdb-amd64:v1.3.3# docker pull k8s.gcr.io/heapster-amd64:v1.4.2
1、k8s-dashborad-sa.yaml文件，secrct和serviceaccount配置
# cat k8s-dashborad-sa.yaml # ------------------- Dashboard Secret ------------------- #apiVersion: v1kind: Secretmetadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard-certs namespace: kube-systemtype: Opaque---# ------------------- Dashboard Service Account ------------------- #apiVersion: v1kind: ServiceAccountmetadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard namespace: kube-system
2、k8s-dashborad-rbac.yaml文件，配置 Role和Role Binding
# cat k8s-dashborad-rbac.yaml # ------------------- Dashboard Role & Role Binding ------------------- #kind: RoleapiVersion: rbac.authorization.k8s.io/v1metadata: name: kubernetes-dashboard-minimal namespace: kube-systemrules: # Allow Dashboard to create 'kubernetes-dashboard-key-holder' secret.- apiGroups: [""] resources: ["secrets"] verbs: ["create"] # Allow Dashboard to create 'kubernetes-dashboard-settings' config map.- apiGroups: [""] resources: ["configmaps"] verbs: ["create"] # Allow Dashboard to get, update and delete Dashboard exclusive secrets.- apiGroups: [""] resources: ["secrets"] resourceNames: ["kubernetes-dashboard-key-holder", "kubernetes-dashboard-certs"] verbs: ["get", "update", "delete"] # Allow Dashboard to get and update 'kubernetes-dashboard-settings' config map.- apiGroups: [""] resources: ["configmaps"] resourceNames: ["kubernetes-dashboard-settings"] verbs: ["get", "update"] # Allow Dashboard to get metrics from heapster.- apiGroups: [""] resources: ["services"] resourceNames: ["heapster"] verbs: ["proxy"] - apiGroups: [""] resources: ["services/proxy"] resourceNames: ["heapster", "http:heapster:", "https:heapster:"] verbs: ["get"] ---apiVersion: rbac.authorization.k8s.io/v1kind: RoleBindingmetadata: name: kubernetes-dashboard-minimal namespace: kube-systemroleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: kubernetes-dashboard-minimalsubjects:- kind: ServiceAccount name: kubernetes-dashboard namespace: kube-system
3、k8s-dashborad-deployment.yaml配置文件，定義創(chuàng)建pod的模板和副本數(shù)
# cat k8s-dashborad-deployment.yaml # ------------------- Dashboard Deployment ------------------- #kind: DeploymentapiVersion: apps/v1beta2metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kube-systemspec:
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
k8s-app: kubernetes-dashboard
template:
metadata:
labels:
k8s-app: kubernetes-dashboard
spec:
containers:
- name: kubernetes-dashboard
image: k8s.gcr.io/kubernetes-dashboard-amd64:v1.8.3
ports:
- containerPort: 8443
protocol: TCP
args:
- --auto-generate-certificates # Uncomment the following line to manually specify Kubernetes API server Host
# If not specified, Dashboard will attempt to auto discover the API server and connect
# to it. Uncomment only if the default does not work.
# - --apiserver-host=http://my-address:port
volumeMounts:
- name: kubernetes-dashboard-certs
mountPath: /certs # Create on-disk volume to store exec logs
- mountPath: /tmp
name: tmp-volume
livenessProbe:
httpGet:
scheme: HTTPS
path: /
port: 8443
initialDelaySeconds: 30
timeoutSeconds: 30
volumes:
- name: kubernetes-dashboard-certs
secret:
secretName: kubernetes-dashboard-certs
- name: tmp-volume
emptyDir: {}
serviceAccountName: kubernetes-dashboard # Comment the following tolerations if Dashboard must not be deployed on master
tolerations:
- key: node-role.kubernetes.io/master
effect: NoSchedule4、 k8s-dashborad-service.yaml配置文件，定義service
# cat k8s-dashborad-service.yaml # ------------------- Dashboard Service ------------------- #kind: ServiceapiVersion: v1metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard namespace: kube-systemspec: ports: - port: 443 targetPort: 8443 nodePort: 8490 type: NodePort selector: k8s-app: kubernetes-dashboard
# kubectl create -f .# kubectl get pod,deployment,svc -n kube-system
默認(rèn)情況下只支持kubeconfig和令牌認(rèn)證
# echo 'admin,admin,1' > /etc/kubernetes/basic_auth_file # grep 'auth' /usr/lib/systemd/system/kube-apiserver.service --authorization-mode=Node,RBAC \ --runtime-config=rbac.authorization.k8s.io/v1alpha1 \ --enable-bootstrap-token-auth=true \ --token-auth-file=/etc/kubernetes/token.csv \ --basic-auth-file=/etc/kubernetes/basic_auth_file \# grep ‘basic’ k8s-dashborad-deployment.yaml （配置在args下面） - --authentication-mode=basic# systemctl daemon-reload# systemctl restart kube-apiserver # kubectl apply -f k8s-dashborad-deployment.yaml
將admin用戶和cluter-admin role進(jìn)行角色綁定
# curl --insecure https://vm1:6443 -basic -u admin:admin # kubectl create clusterrolebinding \login-on-dashboard-with-cluster-admin \ --clusterrole=cluster-admin --user=admin# curl --insecure https://vm1:6443 -basic -u admin:admin
在沒有配置heapster和influxdb的情況下，pod的metric信息是無法獲取到的，而早前版本K8S的HPA特性依賴的metric數(shù)據(jù)來源恰巧就是heapster和influxdb。
1、準(zhǔn)備yaml配置文件
# cat heapster-sa.yaml apiVersion: v1kind: ServiceAccountmetadata: name: heapster namespace: kube-system
# cat heapster-rbac.yaml kind: ClusterRoleBindingapiVersion: rbac.authorization.k8s.io/v1beta1metadata: name: heapsterroleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:heapstersubjects:- kind: ServiceAccount name: heapster namespace: kube-system
# cat heapster-deployment.yaml apiVersion: extensions/v1beta1 kind: Deployment metadata: name: heapster namespace: kube-system spec: replicas: 1 template: metadata: labels: task: monitoring k8s-app: heapster spec: serviceAccountName: heapster containers: - name: heapster image: k8s.gcr.io/heapster-amd64:v1.4.2 imagePullPolicy: IfNotPresent command: - /heapster - --source=kubernetes:https://kubernetes.default - --sink=influxdb:http://monitoring-influxdb.kube-system.svc:8086
# cat heapster-service.yaml apiVersion: v1kind: Servicemetadata: labels: task: monitoring kubernetes.io/cluster-service: 'true' kubernetes.io/name: Heapster name: heapster namespace: kube-systemspec: ports: - port: 80 targetPort: 8082 selector:k8s-app: heapster
# cat influxdb-deployment.yaml apiVersion: extensions/v1beta1kind: Deploymentmetadata:
name: monitoring-influxdb
namespace: kube-systemspec:
replicas: 1
template:
metadata:
labels:
task: monitoring
k8s-app: influxdb
spec:
containers:
- name: influxdb
image: k8s.gcr.io/heapster-influxdb-amd64:v1.3.3
volumeMounts:
- mountPath: /data
name: influxdb-storage
volumes:
- name: influxdb-storage
emptyDir: {}# cat influxdb-service.yaml apiVersion: v1kind: Servicemetadata: labels: task: monitoring kubernetes.io/cluster-service: 'true' kubernetes.io/name: monitoring-influxdb name: monitoring-influxdb namespace: kube-systemspec: ports: - port: 8086 targetPort: 8086 selector: k8s-app: influxdb
獲取heapster中的獲取支持的metrics
# kubectl run -i --tty curl --namespace=kube-system \--image=registry.59iedu.com/webwurst/curl-utils /bin/sh # curl http://heapster/api/v1/model/metrics# curl http://heapster/api/v1/model/debug/allkeys
# kubectl get node # kubectl top node
當(dāng)heapster和influxdb pod都正常運(yùn)行的時(shí)候，在dashboard里面就可以看到CPU和內(nèi)存的監(jiān)控?cái)?shù)據(jù)了。
1、刪除apiserver里面basic認(rèn)證相關(guān)的配置后重啟apiserver
--basic-auth-file=/etc/kubernetes/basic_auth_file
# systemctl daemon-reload# systemctl restart kube-apiserver
2、刪除clusterrolebinding
# kubectl delete clusterrolebinding login-on-dashboard-with-cluster-admin
3、修改k8s-dashborad-deployment.yaml文件
去掉- --authentication-mode=basic參數(shù)
4、創(chuàng)建普通用戶,賦予所有namespace下資源的get、watch和list權(quán)限。
這里通過clusterrole和culsterrolebinding賦予所有namespace相關(guān)資源的get、watch、list權(quán)限，實(shí)際應(yīng)用環(huán)境建議使用創(chuàng)建role和rolebinding指定特定的namespace相關(guān)資源權(quán)限，各資源權(quán)限的賦予規(guī)則遵循最小權(quán)限原則。
# cat rbac-yang.yaml kind: ClusterRoleapiVersion: rbac.authorization.k8s.io/v1metadata: name: role-yangrules:- apiGroups: [""] resources: ["*"] verbs: ["get","watch","list" ] - apiGroups: ["storage.k8s.io"] resources: ["*"] verbs: ["get","watch","list" ] - apiGroups: ["rbac.authorization.k8s.io"] resources: ["*"] verbs: ["get","watch","list" ] - apiGroups: ["batch"] resources: ["*"] verbs: ["get","watch","list" ] - apiGroups: ["apps"] resources: ["*"] verbs: ["get","watch","list" ] - apiGroups: ["extensions"] resources: ["*"] verbs: ["get","watch","list" ] ---kind: ClusterRoleBindingapiVersion: rbac.authorization.k8s.io/v1metadata: name: role-bind-yangsubjects:- kind: ServiceAccount name: yang namespace: kube-systemroleRef: kind: ClusterRole name: role-yang apiGroup: rbac.authorization.k8s.io
# kubectl create sa yang -n kube-system# kubectl create -f rbac-yang.yaml # kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep yang | awk '{print $1}')
5、測(cè)試普通用戶的權(quán)限
6、創(chuàng)建super用戶admin
# kubectl create sa admin -n kube-system# cat rbac-admin.yaml apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: admin
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: admin
namespace: kube-system# kubectl create -f rbac-admin.yaml # kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep admin | awk '{print $1}')
使用admin用戶的token登陸后繼承cluster-admin的權(quán)限
參考：
https://github.com/kubernetes/dashboard/wiki/Creating-sample-user
https://github.com/kubernetes/dashboard/wiki/Access-control
https://github.com/kubernetes/heapster/blob/master/docs/model.md
免責(zé)聲明：本站發(fā)布的內(nèi)容（圖片、視頻和文字）以原創(chuàng)、轉(zhuǎn)載和分享為主，文章觀點(diǎn)不代表本網(wǎng)站立場(chǎng)，如果涉及侵權(quán)請(qǐng)聯(lián)系站長郵箱：is@yisu.com進(jìn)行舉報(bào)，并提供相關(guān)證據(jù)，一經(jīng)查實(shí)，將立刻刪除涉嫌侵權(quán)內(nèi)容。
億速云公眾號(hào)
手機(jī)網(wǎng)站二維碼