Kubernets 部署 Harbor （最新版）

容器，鏡像，倉(cāng)庫(kù)號(hào)稱(chēng)容器三大基本組件，玩轉(zhuǎn)K8S自然逃脫不了鏡像倉(cāng)庫(kù)搭建的宿命，私有鏡像倉(cāng)庫(kù)的必要性我想沒(méi)必要在這里重申。今天這篇文章，在實(shí)驗(yàn)環(huán)境下介紹K8S里面完整部署一個(gè)私有的harbor鏡像倉(cāng)庫(kù)的搭建過(guò)程。

K8S一定要用Harbor作為鏡像倉(cāng)庫(kù)嗎？當(dāng)然不一定，但是通過(guò)對(duì)比你會(huì)知道，無(wú)論從哪方面Harbor正努力并已經(jīng)成了你幾乎唯一的選擇，就像K8S作為容器編排的事實(shí)上的標(biāo)準(zhǔn)一樣，你幾乎沒(méi)有第二個(gè)更好的選擇。

這也是筆者苦心琢磨，并一定要將其部署成功并撰寫(xiě)此文奉獻(xiàn)給讀者的目的。

廢話少說(shuō)，言歸正傳，介紹實(shí)驗(yàn)環(huán)境：

1，CentOS 7 minimal

2, 單節(jié)點(diǎn)的K8S master 1.15.5 ；（由于1.16改動(dòng)較大，所有啟用1.15的最高版本）

3，helm 2.15

4，harbor

helm部署
一、Helm 客戶端安裝

Helm 的安裝方式很多，這里采用二進(jìn)制的方式安裝。更多安裝方法可以參考 Helm 的官方幫助文檔。

方式一：使用官方提供的腳本一鍵安裝

curl?https://raw.githubusercontent.com/helm/helm/master/scripts/get?>?get_helm.sh
chmod?700?get_helm.sh
./get_helm.sh

二、Helm 服務(wù)端安裝Tiller

注意：先在 K8S 集群上每個(gè)節(jié)點(diǎn)安裝 socat 軟件(yum install -y socat )，不然會(huì)報(bào)如下錯(cuò)誤：

error?forwarding?port?44134?to?pod?dc6da4ab99ad9c497c0cef1776b9dd18e0a612d507e2746ed63d36ef40f30174,?uid?:?unable?to?do?port?forwarding:?socat?not?found.
Error:?cannot?connect?to?Tiller

centos7 是默認(rèn)安裝，所以我這里忽略，請(qǐng)確認(rèn)安裝。

Tiller 是以 Deployment 方式部署在 Kubernetes 集群中的，只需使用以下指令便可簡(jiǎn)單的完成安裝：

helm?init

三、給 Tiller 授權(quán)

因?yàn)?Helm 的服務(wù)端 Tiller 是一個(gè)部署在 Kubernetes 中 Kube-System Namespace 下 的 Deployment，它會(huì)去連接 Kube-Api 在 Kubernetes 里創(chuàng)建和刪除應(yīng)用。
而從 Kubernetes 1.6 版本開(kāi)始，API Server 啟用了 RBAC 授權(quán)。目前的 Tiller 部署時(shí)默認(rèn)沒(méi)有定義授權(quán)的 ServiceAccount，這會(huì)導(dǎo)致訪問(wèn) API Server 時(shí)被拒絕。所以我們需要明確為 Tiller 部署添加授權(quán)。
為 Tiller創(chuàng)建 Kubernetes 的服務(wù)帳號(hào)和綁定角色 ：

kubectl?create?serviceaccount?--namespace?kube-system?tiller
kubectl?create?clusterrolebinding?tiller-cluster-rule?--clusterrole=cluster-admin?--serviceaccount=kube-system:tiller

使用 kubectl patch 更新 API 對(duì)象?? ：

kubectl?patch?deploy?--namespace?kube-system?tiller-deploy?-p?'{"spec":{"template":{"spec":{"serviceAccount":"tiller"}}}}'

查看是否授權(quán)成功??

kubectl?get?deploy?--namespace?kube-system???tiller-deploy??--output?yaml|grep??serviceAccount
????serviceAccount:?tiller
????serviceAccountName:?tiller

四、驗(yàn)證 Tiller 是否安裝成功???

kubectl?-n?kube-system?get?pods|grep?tiller
tiller-deploy-6d68f5c78f-nql2z??????????1/1???????Running???0??????????5m

helm?version
Client:?&version.Version{SemVer:"v2.15.0",?GitCommit:"c2440264ca6c078a06e088a838b0476d2fc14750",?GitTreeState:"clean"}
Server:?&version.Version{SemVer:"v2.15.0",?GitCommit:"c2440264ca6c078a06e088a838b0476d2fc14750",?GitTreeState:"clean"}

harbor安裝

具體可以看看官方的介紹https://github.com/goharbor/harbor-helm
添加helm倉(cāng)庫(kù)：

helm?repo?add?harbor?https://helm.goharbor.io

官方的介紹教程是假設(shè)各位都是高手（我這里心里默默問(wèn)候它），下面介紹點(diǎn)基本的詳細(xì)操作：

一，搜索harbor chart 項(xiàng)目：

helm?search?harbor

二，下載到本地，便于修改values.yaml：

helm?fetch?harbor/harbor

解壓下載的項(xiàng)目包，并進(jìn)入解壓路徑修改values.yaml文件：

?tar?zxvf?harbor-1.2.1.tgz?
?cd?harbor
?vim?values.yaml

可以參考官方介紹修改參數(shù)，但是對(duì)于初學(xué)者除了數(shù)據(jù)持久化需要修改，其他一律默認(rèn)，后面熟悉了再逐一修改：

更改values.yaml所有的storageClass為storageClass: "nfs"，這是我已經(jīng)提前部署好的，

如果你錯(cuò)過(guò)了，可以回去看我的教程《初探Kubernetes動(dòng)態(tài)卷存儲(chǔ)(NFS)》，把它補(bǔ)上：https://blog.51cto.com/kingda/2440315；

當(dāng)然你可以一條語(yǔ)句直接修改此文件：

sed?-i?'s#storageClass:?""#storageClass:?"nfs"#g'?values.yaml

其他地方一律默認(rèn)，然后開(kāi)始安裝：

helm?install?--name?harbor-v1?.??--wait?--timeout?1500?--debug?--namespace?harbor

由于PV和PVC的自動(dòng)創(chuàng)建工作可能沒(méi)你想象的那么快，所以導(dǎo)致很多pod開(kāi)始會(huì)報(bào)錯(cuò)，所以一定要有點(diǎn)耐心等待它自動(dòng)多次重啟就緒。

上面那條安裝命令可能一直卡在執(zhí)行狀態(tài)，請(qǐng)一定要有點(diǎn)耐心，等待所有pod都啟動(dòng)成功，helm才會(huì)檢測(cè)到所有pod的安裝狀態(tài)并執(zhí)行完畢。

由于我們是才用默認(rèn)設(shè)置安裝，所以helm默認(rèn)是啟動(dòng)ingress的方式暴露harbor服務(wù)，所以如果你沒(méi)有提前安裝ingress控制器的話，雖然不影響harbor正常運(yùn)行但是你無(wú)法訪問(wèn)它，

所以，下面介紹安裝ingress控制器：

K8S官方有源碼介紹，這里直接貼出一鍵安裝腳本文件：

apiVersion:?v1
kind:?Namespace
metadata:
??name:?ingress-nginx
??labels:
????app.kubernetes.io/name:?ingress-nginx
????app.kubernetes.io/part-of:?ingress-nginx
---
kind:?ConfigMap
apiVersion:?v1
metadata:
??name:?nginx-configuration
??namespace:?ingress-nginx
??labels:
????app.kubernetes.io/name:?ingress-nginx
????app.kubernetes.io/part-of:?ingress-nginx
---
kind:?ConfigMap
apiVersion:?v1
metadata:
??name:?tcp-services
??namespace:?ingress-nginx
??labels:
????app.kubernetes.io/name:?ingress-nginx
????app.kubernetes.io/part-of:?ingress-nginx
---
kind:?ConfigMap
apiVersion:?v1
metadata:
??name:?udp-services
??namespace:?ingress-nginx
??labels:
????app.kubernetes.io/name:?ingress-nginx
????app.kubernetes.io/part-of:?ingress-nginx
---
apiVersion:?v1
kind:?ServiceAccount
metadata:
??name:?nginx-ingress-serviceaccount
??namespace:?ingress-nginx
??labels:
????app.kubernetes.io/name:?ingress-nginx
????app.kubernetes.io/part-of:?ingress-nginx
---
apiVersion:?rbac.authorization.k8s.io/v1beta1
kind:?ClusterRole
metadata:
??name:?nginx-ingress-clusterrole
??labels:
????app.kubernetes.io/name:?ingress-nginx
????app.kubernetes.io/part-of:?ingress-nginx
rules:
??-?apiGroups:
??????-?""
????resources:
??????-?configmaps
??????-?endpoints
??????-?nodes
??????-?pods
??????-?secrets
????verbs:
??????-?list
??????-?watch
??-?apiGroups:
??????-?""
????resources:
??????-?nodes
????verbs:
??????-?get
??-?apiGroups:
??????-?""
????resources:
??????-?services
????verbs:
??????-?get
??????-?list
??????-?watch
??-?apiGroups:
??????-?"extensions"
????resources:
??????-?ingresses
????verbs:
??????-?get
??????-?list
??????-?watch
??-?apiGroups:
??????-?""
????resources:
??????-?events
????verbs:
??????-?create
??????-?patch
??-?apiGroups:
??????-?"extensions"
????resources:
??????-?ingresses/status
????verbs:
??????-?update
---
apiVersion:?rbac.authorization.k8s.io/v1beta1
kind:?Role
metadata:
??name:?nginx-ingress-role
??namespace:?ingress-nginx
??labels:
????app.kubernetes.io/name:?ingress-nginx
????app.kubernetes.io/part-of:?ingress-nginx
rules:
??-?apiGroups:
??????-?""
????resources:
??????-?configmaps
??????-?pods
??????-?secrets
??????-?namespaces
????verbs:
??????-?get
??-?apiGroups:
??????-?""
????resources:
??????-?configmaps
????resourceNames:
??????#?Defaults?to?"<election-id>-<ingress-class>"
??????#?Here:?"<ingress-controller-leader>-<nginx>"
??????#?This?has?to?be?adapted?if?you?change?either?parameter
??????#?when?launching?the?nginx-ingress-controller.
??????-?"ingress-controller-leader-nginx"
????verbs:
??????-?get
??????-?update
??-?apiGroups:
??????-?""
????resources:
??????-?configmaps
????verbs:
??????-?create
??-?apiGroups:
??????-?""
????resources:
??????-?endpoints
????verbs:
??????-?get
---
apiVersion:?rbac.authorization.k8s.io/v1beta1
kind:?RoleBinding
metadata:
??name:?nginx-ingress-role-nisa-binding
??namespace:?ingress-nginx
??labels:
????app.kubernetes.io/name:?ingress-nginx
????app.kubernetes.io/part-of:?ingress-nginx
roleRef:
??apiGroup:?rbac.authorization.k8s.io
??kind:?Role
??name:?nginx-ingress-role
subjects:
??-?kind:?ServiceAccount
????name:?nginx-ingress-serviceaccount
????namespace:?ingress-nginx
---
apiVersion:?rbac.authorization.k8s.io/v1beta1
kind:?ClusterRoleBinding
metadata:
??name:?nginx-ingress-clusterrole-nisa-binding
??labels:
????app.kubernetes.io/name:?ingress-nginx
????app.kubernetes.io/part-of:?ingress-nginx
roleRef:
??apiGroup:?rbac.authorization.k8s.io
??kind:?ClusterRole
??name:?nginx-ingress-clusterrole
subjects:
??-?kind:?ServiceAccount
????name:?nginx-ingress-serviceaccount
????namespace:?ingress-nginx
---
apiVersion:?extensions/v1beta1
kind:?DaemonSet
metadata:
??name:?nginx-ingress-controller
??namespace:?ingress-nginx
??labels:
????app.kubernetes.io/name:?ingress-nginx
????app.kubernetes.io/part-of:?ingress-nginx
spec:
??#replicas:?1
??selector:
????matchLabels:
??????app.kubernetes.io/name:?ingress-nginx
??????app.kubernetes.io/part-of:?ingress-nginx
??updateStrategy:
????rollingUpdate:
??????maxUnavailable:?1
????type:?RollingUpdate
??template:
????metadata:
??????labels:
????????app.kubernetes.io/name:?ingress-nginx
????????app.kubernetes.io/part-of:?ingress-nginx
??????annotations:
????????prometheus.io/port:?"10254"
????????prometheus.io/scrape:?"true"
????spec:
??????serviceAccountName:?nginx-ingress-serviceaccount
??????hostNetwork:?true
??????containers:
????????-?name:?nginx-ingress-controller
??????????image:?quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.22.0
??????????args:
????????????-?/nginx-ingress-controller
????????????-?--configmap=$(POD_NAMESPACE)/nginx-configuration
????????????-?--tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
????????????-?--udp-services-configmap=$(POD_NAMESPACE)/udp-services
????????????-?--publish-service=$(POD_NAMESPACE)/ingress-nginx
????????????-?--annotations-prefix=nginx.ingress.kubernetes.io
??????????securityContext:
????????????allowPrivilegeEscalation:?true
????????????capabilities:
??????????????drop:
????????????????-?ALL
??????????????add:
????????????????-?NET_BIND_SERVICE
????????????#?www-data?->?33
????????????runAsUser:?33
??????????env:
????????????-?name:?POD_NAME
??????????????valueFrom:
????????????????fieldRef:
??????????????????fieldPath:?metadata.name
????????????-?name:?POD_NAMESPACE
??????????????valueFrom:
????????????????fieldRef:
??????????????????fieldPath:?metadata.namespace
??????????ports:
????????????-?name:?http
??????????????containerPort:?80
????????????-?name:?https
??????????????containerPort:?443
??????????livenessProbe:
????????????failureThreshold:?3
????????????httpGet:
??????????????path:?/healthz
??????????????port:?10254
??????????????scheme:?HTTP
????????????initialDelaySeconds:?10
????????????periodSeconds:?10
????????????successThreshold:?1
????????????timeoutSeconds:?1
??????????readinessProbe:
????????????failureThreshold:?3
????????????httpGet:
??????????????path:?/healthz
??????????????port:?10254
??????????????scheme:?HTTP
????????????periodSeconds:?10
????????????successThreshold:?1
????????????timeoutSeconds:?1
---

使用kubectl 安裝即可。

如果你已經(jīng)解析默認(rèn)的ingress訪問(wèn)域名到K8S的任意節(jié)點(diǎn)上，那么直接使用默認(rèn)賬號(hào)和密碼登錄即可。