您好,登錄后才能下訂單哦!
本篇內(nèi)容主要講解“Docker私有倉庫Harbor怎么部署”,感興趣的朋友不妨來看看。本文介紹的方法操作簡(jiǎn)單快捷,實(shí)用性強(qiáng)。下面就讓小編來帶大家學(xué)習(xí)“Docker私有倉庫Harbor怎么部署”吧!
一、Harbor倉庫介紹
我們?cè)谌粘ocker容器使用和管理過程中,漸漸發(fā)現(xiàn)部署企業(yè)私有倉庫往往是很有必要的, 它可以幫助你管理企業(yè)的一些敏感鏡像, 同時(shí)由于Docker Hub的下載速度和GFW的原因, 往往需要將一些無法直接下載的鏡像導(dǎo)入本地私有倉庫. 而Harbor就是部署企業(yè)私有倉庫的一個(gè)不二之選。
Harbor是由VMware公司開源的企業(yè)級(jí)的Docker Registry管理項(xiàng)目,Harbor主要提供Dcoker Registry管理UI,提供的功能包括:基于角色訪問的控制權(quán)限管理(RBAC)、AD/LDAP集成、日志審核、管理界面、自我注冊(cè)、鏡像復(fù)制和中文支持等。
Harbor的目標(biāo)是幫助用戶迅速搭建一個(gè)企業(yè)級(jí)的Docker registry服務(wù)。它以Docker公司開源的registry為基礎(chǔ),額外提供了如下功能:
-> 基于角色的訪問控制(Role Based Access Control)
-> 基于策略的鏡像復(fù)制(Policy based image replication)
-> 鏡像的漏洞掃描(Vulnerability Scanning)
-> AD/LDAP集成(LDAP/AD support)
-> 鏡像的刪除和空間清理(Image deletion & garbage collection)
-> 友好的管理UI(Graphical user portal)
-> 審計(jì)日志(Audit logging)
-> RESTful API
-> 部署簡(jiǎn)單(Easy deployment)
Harbor的所有組件都在Dcoker中部署,所以Harbor可使用Docker Compose快速部署。需要特別注意:由于Harbor是基于Docker Registry V2版本,所以docker必須大于等于1.10.0版本,docker-compose必須要大于1.6.0版本!
Harbor的每個(gè)組件都是以Docker容器的形式構(gòu)建的,可以使用Docker Compose來進(jìn)行部署。如果環(huán)境中使用了kubernetes,Harbor也提供了kubernetes的配置文件。Harbor大概需要以下幾個(gè)容器組成:ui(Harbor的核心服務(wù))、log(運(yùn)行著rsyslog的容器,進(jìn)行日志收集)、mysql(由官方mysql鏡像構(gòu)成的數(shù)據(jù)庫容器)、Nginx(使用Nginx做反向代理)、registry(官方的Docker registry)、adminserver(Harbor的配置數(shù)據(jù)管理器)、jobservice(Harbor的任務(wù)管理服務(wù))、redis(用于存儲(chǔ)session)。
Harbor是一個(gè)用于存儲(chǔ)和分發(fā)Docker鏡像的企業(yè)級(jí)Registry服務(wù)器,整體架構(gòu)還是很清晰的。下面借用了網(wǎng)上的架構(gòu)圖:
-> Nginx(即Proxy代理層): Nginx前端代理,主要用于分發(fā)前端頁面ui訪問和鏡像上傳和下載流量; Harbor的registry,UI,token等服務(wù),通過一個(gè)前置的反向代理統(tǒng)一接收瀏覽器、Docker客戶端的請(qǐng)求,并將請(qǐng)求轉(zhuǎn)發(fā)給后端不同的服務(wù)。
-> Registry v2: 鏡像倉庫,負(fù)責(zé)存儲(chǔ)鏡像文件; Docker官方鏡像倉庫, 負(fù)責(zé)儲(chǔ)存Docker鏡像,并處理docker push/pull命令。由于我們要對(duì)用戶進(jìn)行訪問控制,即不同用戶對(duì)Docker image有不同的讀寫權(quán)限,Registry會(huì)指向一個(gè)token服務(wù),強(qiáng)制用戶的每次docker pull/push請(qǐng)求都要攜帶一個(gè)合法的token, Registry會(huì)通過公鑰對(duì)token進(jìn)行解密驗(yàn)證。
-> Database(MySQL或Postgresql):為core services提供數(shù)據(jù)庫服務(wù),負(fù)責(zé)儲(chǔ)存用戶權(quán)限、審計(jì)日志、Docker image分組信息等數(shù)據(jù)。
-> Core services(Admin Server): 這是Harbor的核心功能,主要提供以下服務(wù):
-> UI:提供圖形化界面,幫助用戶管理registry上的鏡像(image), 并對(duì)用戶進(jìn)行授權(quán)。
-> webhook:為了及時(shí)獲取registry 上image狀態(tài)變化的情況, 在Registry上配置webhook,把狀態(tài)變化傳遞給UI模塊。
-> Auth服務(wù):負(fù)責(zé)根據(jù)用戶權(quán)限給每個(gè)docker push/pull命令簽發(fā)token. Docker 客戶端向Regiøstry服務(wù)發(fā)起的請(qǐng)求,如果不包含token,會(huì)被重定向到這里,獲得token后再重新向Registry進(jìn)行請(qǐng)求。
-> API: 提供Harbor RESTful API
-> Replication Job Service:提供多個(gè) Harbor 實(shí)例之間的鏡像同步功能。
-> Log collector:為了幫助監(jiān)控Harbor運(yùn)行,負(fù)責(zé)收集其他組件的log,供日后進(jìn)行分析。
再來仔細(xì)看下Harbor主要組件和數(shù)據(jù)流走向:
-> proxy,它是一個(gè)nginx前端代理,主要是分發(fā)前端頁面ui訪問和鏡像上傳和下載流量,上圖中通過深藍(lán)色先標(biāo)識(shí);
-> ui提供了一個(gè)web管理頁面,當(dāng)然還包括了一個(gè)前端頁面和后端API,底層使用mysql數(shù)據(jù)庫;
-> registry是鏡像倉庫,負(fù)責(zé)存儲(chǔ)鏡像文件,當(dāng)鏡像上傳完畢后通過hook通知ui創(chuàng)建repository,上圖通過紅色線標(biāo)識(shí),當(dāng)然registry的token認(rèn)證也是通過ui組件完成;
-> adminserver是系統(tǒng)的配置管理中心附帶檢查存儲(chǔ)用量,ui和jobserver啟動(dòng)時(shí)候回需要加載adminserver的配置,通過灰色線標(biāo)識(shí);
-> jobsevice是負(fù)責(zé)鏡像復(fù)制工作的,他和registry通信,從一個(gè)registry pull鏡像然后push到另一個(gè)registry,并記錄job_log,上圖通過紫色線標(biāo)識(shí);
-> log是日志匯總組件,通過docker的log-driver把日志匯總到一起,通過淺藍(lán)色線條標(biāo)識(shí)。
誤區(qū)一: Harbor是負(fù)責(zé)存儲(chǔ)容器鏡像的 (Harbor是鏡像倉庫,那么它就應(yīng)當(dāng)是存儲(chǔ)鏡像的)
其實(shí)關(guān)于鏡像的存儲(chǔ),Harbor使用的是官方的docker registry服務(wù)去完成,至于registry是用本地存儲(chǔ)或者s3都是可以的,Harbor的功能是在此之上提供用戶權(quán)限管理、鏡像復(fù)制等功能,提高使用的registry的效率。誤區(qū)二:Harbor鏡像復(fù)制是存儲(chǔ)直接復(fù)制 (鏡像的復(fù)制,很多人以為應(yīng)該是鏡像分層文件的直接拷貝)
其實(shí)Harbor鏡像復(fù)制采用了一個(gè)更加通用、高屋建瓴的做法,通過docker registry 的API去拷貝,這不是省事,這種做法屏蔽了繁瑣的底層文件操作、不僅可以利用現(xiàn)有docker registry功能不必重復(fù)造輪子,而且可以解決沖突和一致性的問題。
這里不建議使用kubernetes來部署, 原因是鏡像倉庫非常重要, 盡量保證部署和維護(hù)的簡(jiǎn)潔性, 因此這里直接使用compose的方式進(jìn)行部署。官方提供3種部署Harbor的方式:
1)在線安裝: 從Docker Hub下載Harbor的鏡像來安裝, 由于Docker Hub比較慢, 建議Docker配置好加速器。
2)離線安裝: 這種方式應(yīng)對(duì)與部署主機(jī)沒聯(lián)網(wǎng)的情況使用。需要提前下載離線安裝包: harbor-offline-installer-.tgz 到本地
3)OVA安裝: 這個(gè)主要用vCentor環(huán)境是使用
后面部署時(shí)會(huì)為Docker配置鏡像加速器, 因此會(huì)采用在線部署的方式, 部署步驟如下:
-> 下載Harbor最新的在線安裝包
-> 配置Harbor (harbor.cfg)
-> 運(yùn)行install.sh來安裝和啟動(dòng)Harbor
-> Harbor的日志路徑:/var/log/harbor
Harbor倉庫部署的官方要求的最小系統(tǒng)配置
-> 2個(gè)cpu
-> 4g內(nèi)存
-> 40g硬盤,因?yàn)槭谴鎯?chǔ)鏡像的所以推薦硬盤大點(diǎn)。
Harbor GitHub
Harbor 安裝手冊(cè)
Harbor 用戶手冊(cè)
Harbor以容器的形式進(jìn)行部署, 因此可以被部署到任何支持Docker的Linux發(fā)行版, 要使用Harbor,需要安裝docker和docker-compose編排工具,并且具備如下環(huán)境: Python2.7+ Docker Engine 1.10+ Docker Compose 1.6.0+ 這里測(cè)試環(huán)境部署到Centos7.5機(jī)器上,如下: [root@harbor-node ~]# cat /etc/redhat-release CentOS Linux release 7.6.1810 (Core) [root@harbor-node ~]# setenforce 0 [root@harbor-node ~]# cat /etc/sysconfig/selinux ........... SELINUX=disabled [root@harbor-node ~]# systemctl stop firewalld [root@harbor-node ~]# systemctl disable firewalld [root@harbor-node ~]# firewall-cmd --state not running centos7自帶的python版本就是2.7.5 [root@harbor-node ~]# python --version Python 2.7.5
更新yum包 [root@harbor-node ~]# yum update 卸載舊版本 Docker [root@harbor-node ~]# yum remove docker docker-common docker-selinux docker-engine 安裝軟件包 [root@harbor-node ~]# yum install -y yum-utils device-mapper-persistent-data lvm2 添加 Docker yum源 [root@harbor-node ~]# yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo Loaded plugins: fastestmirror adding repo from: https://download.docker.com/linux/centos/docker-ce.repo grabbing file https://download.docker.com/linux/centos/docker-ce.repo to /etc/yum.repos.d/docker-ce.repo repo saved to /etc/yum.repos.d/docker-ce.repo 安裝 Docker (直接yum按照docker-ce即可) 隨著Docker的不斷流行與發(fā)展,docker公司(或稱為組織)也開啟了商業(yè)化之路,Docker 從 17.03版本之后分為 CE(Community Edition) 和 EE(Enterprise Edition): 1) Docker EE由公司支持,可在經(jīng)過認(rèn)證的操作系統(tǒng)和云提供商中使用,并可運(yùn)行來自Docker Store的、經(jīng)過認(rèn)證的容器和插件。 2) Docker CE是免費(fèi)的Docker產(chǎn)品的新名稱,Docker CE包含了完整的Docker平臺(tái),非常適合開發(fā)人員和運(yùn)維團(tuán)隊(duì)構(gòu)建容器APP。 事實(shí)上,Docker CE 17.03,可理解為Docker 1.13.1的Bug修復(fù)版本。因此,從Docker 1.13升級(jí)到Docker CE 17.03風(fēng)險(xiǎn)相對(duì)是較小的。 [root@harbor-node ~]# yum -y install docker-ce 啟動(dòng) Docker [root@harbor-node ~]# systemctl start docker [root@harbor-node ~]# systemctl enable docker [root@harbor-node ~]# systemctl status docker ● docker.service - Docker Application Container Engine Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled) Active: active (running) since Sun 2019-05-26 22:15:34 CST; 27min ago Docs: https://docs.docker.com Main PID: 15260 (dockerd) CGroup: /system.slice/docker.service └─15260 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock May 26 22:15:33 harbor-node dockerd[15260]: time="2019-05-26T22:15:33.570826805+08:00" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc420175490, READY" module=grpc May 26 22:15:33 harbor-node dockerd[15260]: time="2019-05-26T22:15:33.570899114+08:00" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc42006de20, READY" module=grpc May 26 22:15:33 harbor-node dockerd[15260]: time="2019-05-26T22:15:33.665440742+08:00" level=info msg="Graph migration to content-addressability took 0.00 seconds" May 26 22:15:33 harbor-node dockerd[15260]: time="2019-05-26T22:15:33.666111994+08:00" level=info msg="Loading containers: start." May 26 22:15:33 harbor-node dockerd[15260]: time="2019-05-26T22:15:33.913110547+08:00" level=info msg="Default bridge (docker0) is assigned with an IP address 172.17.0.0/16... IP address" May 26 22:15:34 harbor-node dockerd[15260]: time="2019-05-26T22:15:34.088687650+08:00" level=info msg="Loading containers: done." May 26 22:15:34 harbor-node dockerd[15260]: time="2019-05-26T22:15:34.128885651+08:00" level=info msg="Docker daemon" commit=481bc77 graphdriver(s)=overlay2 version=18.09.6 May 26 22:15:34 harbor-node dockerd[15260]: time="2019-05-26T22:15:34.129073367+08:00" level=info msg="Daemon has completed initialization" May 26 22:15:34 harbor-node dockerd[15260]: time="2019-05-26T22:15:34.223886566+08:00" level=info msg="API listen on /var/run/docker.sock" May 26 22:15:34 harbor-node systemd[1]: Started Docker Application Container Engine. Hint: Some lines were ellipsized, use -l to show in full. 查看 Docker 版本號(hào) [root@harbor-node ~]# docker --version Docker version 18.09.6, build 481bc77156 [root@harbor-node ~]# docker version Client: Version: 18.09.6 API version: 1.39 Go version: go1.10.8 Git commit: 481bc77156 Built: Sat May 4 02:34:58 2019 OS/Arch: linux/amd64 Experimental: false Server: Docker Engine - Community Engine: Version: 18.09.6 API version: 1.39 (minimum version 1.12) Go version: go1.10.8 Git commit: 481bc77 Built: Sat May 4 02:02:43 2019 OS/Arch: linux/amd64 Experimental: false
Docker Compose 是 Docker 容器進(jìn)行編排的工具,定義和運(yùn)行多容器的應(yīng)用,可以一條命令啟動(dòng)多個(gè)容器。 安裝 epel-release [root@harbor-node ~]# yum install epel-release 安裝 python-pip [root@harbor-node ~]# yum install -y python-pip 安裝 docker-compose [root@harbor-node ~]# pip install docker-compose ......... Successfully installed asn1crypto-0.24.0 bcrypt-3.1.6 cached-property-1.5.1 certifi-2019.3.9 cffi-1.12.3 chardet-3.0.4 cryptography-2.6.1 docker-3.7.2 docker-compose-1.24.0 docker-pycreds-0.4.0 dockerpty-0.4.1 docopt-0.6.2 enum34-1.1.6 functools32-3.2.3.post2 idna-2.7 jsonschema-2.6.0 paramiko-2.4.2 pyasn1-0.4.5 pycparser-2.19 pynacl-1.3.0 requests-2.20.1 texttable-0.9.1 urllib3-1.24.3 websocket-client-0.56.0 You are using pip version 8.1.2, however version 19.1.1 is available. You should consider upgrading via the 'pip install --upgrade pip' command. 查看 docker-compose 版本號(hào) [root@harbor-node ~]# docker-compose -version docker-compose version 1.24.0, build 0aa5906 [root@harbor-node ~]# pip freeze | grep compose You are using pip version 8.1.2, however version 19.1.1 is available. You should consider upgrading via the 'pip install --upgrade pip' command. docker-compose==1.24.0 安裝 git [root@harbor-node ~]# yum install git
[root@harbor-node ~]# mkdir -p /etc/docker [root@harbor-node ~]# cat /etc/docker/daemon.json { "registry-mirrors": ["https://v5d7kh0f.mirror.aliyuncs.com"] }
到Harbor的GitHub倉庫的Release頁面, 下載最新的在線安裝包。
這里分為在線和離線的版本,我下載的是1.8.0在線的版本
[root@harbor-node ~]# ll harbor-online-installer-v1.8.0.tgz -rw-r--r-- 1 root root 7954 May 26 22:45 harbor-online-installer-v1.8.0.tgz [root@harbor-node ~]# tar -zvxf harbor-online-installer-v1.8.0.tgz [root@harbor-node ~]# cd harbor [root@harbor-node harbor]# ls harbor.yml install.sh LICENSE prepare 下載下來之后解壓縮,目錄下會(huì)有harbor.yaml (新版本是.yaml文件,之前版本是.conf 或者 .cfg文件),就是Harbor的配置文件了。 [root@harbor-node harbor]# cp harbor.yml harbor.yml.bak [root@harbor-node harbor]# vim harbor.yml [root@harbor-node harbor]# cat harbor.yml |grep -v "#"|grep -v "^$" hostname: 172.16.60.213 http: port: 80 harbor_admin_password: kevin@BO123 database: password: root123 data_volume: /data clair: updaters_interval: 12 http_proxy: https_proxy: no_proxy: 127.0.0.1,localhost,core,registry jobservice: max_job_workers: 10 chart: absolute_url: disabled log: level: info rotate_count: 50 rotate_size: 200M location: /var/log/harbor _version: 1.8.0 配置解釋 hostname: 修改成Harbao部署機(jī)自身的ip地址 db_password: 這是postgresql數(shù)據(jù)庫root密碼 harbor_admin_password: harbor初始管理員密碼為Harbor12345, 這里最好修改成自己的密碼,默認(rèn)密碼至少8位,最好是大小寫、數(shù)字和特殊字符。 配置完Harbor之后,接著進(jìn)行安裝啟動(dòng)Harbor,Harbor目錄下有一個(gè)install.sh, 執(zhí)行它來進(jìn)行安裝 [root@harbor-node harbor]# ./install.sh ........... ........... ? ----Harbor has been installed and started successfully.---- Now you should be able to visit the admin portal at http://172.16.60.213. For more details, please visit https://github.com/goharbor/harbor . 安裝完成后,會(huì)發(fā)現(xiàn)解壓目錄harbor下面多了一個(gè)docker-compose.yml文件,里面包含了harbor依賴的鏡像和對(duì)應(yīng)容器創(chuàng)建的信息 查看harbor對(duì)應(yīng)容器信息(還可以執(zhí)行"docker images"和"docker ps"查看harbor的鏡像和容器情況) [root@harbor-node harbor]# docker-compose ps #"注意docker-compose"命令只能在當(dāng)前harbor目錄下使用(因?yàn)樵撃夸浵掠衕arbor配置文件) Name Command State Ports --------------------------------------------------------------------------------------------- harbor-core /harbor/start.sh Up (healthy) harbor-db /entrypoint.sh postgres Up (healthy) 5432/tcp harbor-jobservice /harbor/start.sh Up harbor-log /bin/sh -c /usr/local/bin/ ... Up (healthy) 127.0.0.1:1514->10514/tcp harbor-portal nginx -g daemon off; Up (healthy) 80/tcp nginx nginx -g daemon off; Up (healthy) 0.0.0.0:80->80/tcp redis docker-entrypoint.sh redis ... Up 6379/tcp registry /entrypoint.sh /etc/regist ... Up (healthy) 5000/tcp registryctl /harbor/start.sh Up (healthy) 然后就可以訪問harbor了,訪問地址為:http://172.16.60.213 用戶名為admin,密碼為配置文件中定義的"kevin@BO123"
==== 這里需要注意一個(gè)Harbor 部署的坑點(diǎn)(Docker 18.09.1 及以上的版本,系統(tǒng)內(nèi)核版本需要升級(jí)到4.4.x) ====
1)CentOS 7.x 系統(tǒng)自帶的3.10.x內(nèi)核存在一些Bugs,導(dǎo)致運(yùn)行的Docker、Kubernetes不穩(wěn)定。
2)高版本的 docker(1.13 以后) 啟用了3.10 kernel實(shí)驗(yàn)支持的kernel memory account功能(無法關(guān)閉),當(dāng)docker節(jié)點(diǎn)壓力大 (如頻繁啟動(dòng)和停止容器) 時(shí)會(huì)導(dǎo)致 cgroup memory leak;
3)Docker 18.09.1 及以上的版本,需要手動(dòng)升級(jí)內(nèi)核到 4.4.X 以上;
因?yàn)榈贸鼋Y(jié)論:
部署harbor的時(shí)候,要首先查看下本機(jī)的docker版本,如果docker版本在18.90.1以上,則需要手動(dòng)升級(jí)內(nèi)核版本到 4.4.x以上。
否則會(huì)出現(xiàn):
harbor部署Harbor正常啟動(dòng),端口正常監(jiān)聽,防火墻也已關(guān)閉,但是通過http://ip:80 訪問不了harbor,并且/var/log/harbor目錄下沒有任何日志產(chǎn)生!! 使用" telnet ip 80"查看發(fā)現(xiàn)不通或者閃退?。?!
手動(dòng)修改系統(tǒng)內(nèi)核版本可以參考:Kubernetes(K8S)容器集群管理環(huán)境完整部署詳細(xì)教程-上篇
具體升級(jí)內(nèi)核操作如下:
[root@harbor-node ~]# uname -r 3.10.0-862.el7.x86_64 [root@harbor-node ~]# rpm -Uvh http://www.elrepo.org/elrepo-release-7.0-3.el7.elrepo.noarch.rpm 安裝完成后檢查 /boot/grub2/grub.cfg 中對(duì)應(yīng)內(nèi)核 menuentry 中是否包含 initrd16 配置,如果沒有,再安裝一次! [root@harbor-node ~]# yum --enablerepo=elrepo-kernel install -y kernel-lt 設(shè)置開機(jī)從新內(nèi)核啟動(dòng) [root@harbor-node ~]# grub2-set-default 0 重啟機(jī)器 [root@harbor-node ~]# init 6 安裝內(nèi)核源文件(在升級(jí)完內(nèi)核并重啟機(jī)器后執(zhí)行,也可以不用執(zhí)行這一步??蛇x): [root@harbor-node ~]# yum --enablerepo=elrepo-kernel install kernel-lt-devel-$(uname -r) kernel-lt-headers-$(uname -r) [root@harbor-node ~]# uname -r 4.4.180-2.el7.elrepo.x86_64 ======================================================================================================================== 或者也可以采用下面升級(jí)內(nèi)核的方法: # git clone --branch v1.14.1 --single-branch --depth 1 https://github.com/kubernetes/kubernetes # cd kubernetes # KUBE_GIT_VERSION=v1.14.1 ./build/run.sh make kubelet GOFLAGS="-tags=nokmem" # init 6
Harbor 服務(wù)的關(guān)閉和啟動(dòng)
1) Harbor的日志路徑:var/log/harbor [root@harbor-node harbor]# cat harbor.yml|grep log log: # Log files are rotated log_rotate_count times before being removed. If count is 0, old versions are removed rather than rotated. # Log files are rotated only if they grow bigger than log_rotate_size bytes. If size is followed by k, the size is assumed to be in kilobytes. # The directory on your host that store log location: /var/log/harbor [root@harbor-node harbor]# ls /var/log/harbor/ core.log jobservice.log portal.log postgresql.log proxy.log redis.log registryctl.log registry.log 2) 停止和關(guān)閉harbor命令: "docker-compose down -v" [root@harbor-node harbor]# docker-compose down -v Stopping nginx ... done Stopping harbor-jobservice ... done Stopping harbor-portal ... done Stopping harbor-core ... done Stopping redis ... done Stopping registryctl ... done Stopping registry ... done Stopping harbor-db ... done Stopping harbor-log ... done Removing nginx ... done Removing harbor-jobservice ... done Removing harbor-portal ... done Removing harbor-core ... done Removing redis ... done Removing registryctl ... done Removing registry ... done Removing harbor-db ... done Removing harbor-log ... done Removing network harbor_harbor [root@harbor-node harbor]# docker-compose ps Name Command State Ports ------------------------------ [root@harbor-node harbor]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 可以修改harbor配置文件,比如這里修改harbor的web登錄端口,由80端口修改為8080端口 [root@harbor-node harbor]# vim harbor.yml ......... http: # port for http, default is 80. If https enabled, this port will redirect to https port port: 8080 然后將harbor修改的配置更新到 docker-compose.yml 文件 [root@harbor-node harbor]# ./prepare prepare base dir is set to /root/harbor Clearing the configuration file: /config/log/logrotate.conf Clearing the configuration file: /config/nginx/nginx.conf Clearing the configuration file: /config/core/env Clearing the configuration file: /config/core/app.conf Clearing the configuration file: /config/registry/config.yml Clearing the configuration file: /config/registry/root.crt Clearing the configuration file: /config/registryctl/env Clearing the configuration file: /config/registryctl/config.yml Clearing the configuration file: /config/db/env Clearing the configuration file: /config/jobservice/env Clearing the configuration file: /config/jobservice/config.yml Generated configuration file: /config/log/logrotate.conf Generated configuration file: /config/nginx/nginx.conf Generated configuration file: /config/core/env Generated configuration file: /config/core/app.conf Generated configuration file: /config/registry/config.yml Generated configuration file: /config/registryctl/env Generated configuration file: /config/db/env Generated configuration file: /config/jobservice/env Generated configuration file: /config/jobservice/config.yml loaded secret from file: /secret/keys/secretkey Generated configuration file: /compose_location/docker-compose.yml Clean up the input dir 防止容器進(jìn)程沒有權(quán)限讀取生成的配置 [root@harbor-node harbor]# ll common/ total 0 drwxr-xr-x 9 root root 105 May 26 23:10 config [root@harbor-node harbor]# chmod -R 777 common [root@harbor-node harbor]# ll common/ total 0 drwxrwxrwx 9 root root 105 May 27 00:41 config ======================================================================================================================================= 特別注意: 這里的common權(quán)限如果設(shè)置太小,可能會(huì)導(dǎo)致harbor啟動(dòng)后,報(bào)下面的錯(cuò) 發(fā)現(xiàn)啟動(dòng)harbor后,如上有些服務(wù),如nginx,registry狀態(tài)一直是"Restarting",這時(shí)需要查看日志: [root@harbor-node harbor]# tail -100 /var/log/harbor/registry.log |grep error May 27 01:01:18 172.19.0.1 registry[2960]: configuration error: open /etc/registry/config.yml: permission denied May 27 01:01:21 172.19.0.1 registry[2960]: configuration error: open /etc/registry/config.yml: permission denied May 27 01:01:23 172.19.0.1 registry[2960]: configuration error: open /etc/registry/config.yml: permission denied May 27 01:01:27 172.19.0.1 registry[2960]: configuration error: open /etc/registry/config.yml: permission denied ======================================================================================================================================= 最后再次啟動(dòng) harbor [root@harbor-node harbor]# docker-compose up -d Creating network "harbor_harbor" with the default driver Creating harbor-log ... done Creating registryctl ... done Creating registry ... done Creating redis ... done Creating harbor-db ... done Creating harbor-core ... done Creating harbor-jobservice ... done Creating harbor-portal ... done Creating nginx ... done 查看服務(wù) [root@harbor-node harbor]# docker-compose ps Name Command State Ports ------------------------------------------------------------------------------------------------------ harbor-core /harbor/start.sh Up (health: starting) harbor-db /entrypoint.sh postgres Up (health: starting) 5432/tcp harbor-jobservice /harbor/start.sh Up harbor-log /bin/sh -c /usr/local/bin/ ... Up (health: starting) 127.0.0.1:1514->10514/tcp harbor-portal nginx -g daemon off; Up (health: starting) 80/tcp nginx nginx -g daemon off; Up (health: starting) 0.0.0.0:8080->80/tcp redis docker-entrypoint.sh redis ... Up 6379/tcp registry /entrypoint.sh /etc/regist ... Up (health: starting) 5000/tcp registryctl /harbor/start.sh Up (health: starting) 然后訪問http://172.16.60.213:8080,即可訪問harbor的web界面 ############################################################################################### 要是想修改harbor的登陸用戶密碼,則最好在harbor web界面里直接修改,這樣是最保險(xiǎn)的! 如果是想通過修改harbar.yaml文件來重置harbor用戶密碼,則不能單純的修改后就執(zhí)行"./prepare"和重啟docker-compose,這樣是不能修改harbor用戶密碼的! 這時(shí)因?yàn)閔arbor在這里用的是postgresql數(shù)據(jù)庫,以pdkdf2算法保存的秘文密碼!需要先進(jìn)入"harbor-db"容器內(nèi)部,執(zhí)行相關(guān)postgresql命令行。 而且postgresql的用戶密碼采用的是pbkdf2算法,需要提前計(jì)算好新密碼的密鑰值,pdkdf2算法需要"鹽值","迭代次數(shù)"和密鑰長(zhǎng)度int型等,特別麻煩!! 所以如果忘記harbor的web密碼或者是admin密碼需要重置,并且對(duì)于postgresql數(shù)據(jù)庫 或者 pbkdf2算法操作不熟悉的話,建議刪除data源數(shù)據(jù)的database,重新部署! 做法如下: # docker-compose down -v # rm -rf /data/database # vim harbor.yaml #在這里重置或修改密碼 # docker-compose up -d 這樣就可以使用重置后的新密碼登陸harbor web界面了,但是之前創(chuàng)建的用戶和項(xiàng)目就都刪除了。 這種情況最好適用于剛創(chuàng)建初期。 ############################################################################################### docker-compose up -d # 后臺(tái)啟動(dòng),如果容器不存在根據(jù)鏡像自動(dòng)創(chuàng)建 docker-compose down -v # 停止容器并刪除容器 docker-compose start # 啟動(dòng)容器,容器不存在就無法啟動(dòng),不會(huì)自動(dòng)創(chuàng)建鏡像 docker-compose stop # 停止容器 需要注意: 其實(shí)上面是停止docker-compose.yml中定義的所有容器,默認(rèn)情況下docker-compose就是操作同目錄下的docker-compose.yml文件。 如果使用其他yml文件,可以使用-f自己指定。
-> 登錄Harbor web界面,在"系統(tǒng)管理"->"配置管理"->"認(rèn)證模式"->"允許自注冊(cè)"這一項(xiàng)的對(duì)勾去掉,則登錄的時(shí)候就不會(huì)有"用戶注冊(cè)"這一功能了。
-> 可以在"配置管理"這一項(xiàng)進(jìn)行認(rèn)證模式,郵箱,標(biāo)簽等設(shè)置。
1)在harbor遠(yuǎn)程別的客戶機(jī)上登錄 [root@docker-client ~]# docker login 172.16.60.213 Username: admin Password: Error response from daemon: Get https://172.16.60.213/v1/users/: dial tcp 172.16.60.213:443: connect: connection refused 在進(jìn)行harbor登錄或上傳代碼時(shí),會(huì)報(bào)出上面錯(cuò)誤! 這是因?yàn)閐ocker1.3.2版本開始默認(rèn)docker registry使用的是https,而Harbor默認(rèn)設(shè)置的是http方式而不是https,所以當(dāng)執(zhí)行用docker login、pull、push等 命令操作非https的docker regsitry的時(shí)就會(huì)報(bào)錯(cuò)。 解決辦法: 如下,在/etc/docker/daemon.json文件里添加"insecure-registries"配置。(如果還不行,可以嘗試將下面添加的地址由"172.16.60.213"改為"http://172.16.60.213") [root@docker-client ~]# vim /etc/docker/daemon.json { "insecure-registries": [ "172.16.60.213" ] } 然后重啟docker服務(wù) [root@docker-client ~]# systemctl restart docker 接著再次驗(yàn)證harbor登錄,發(fā)現(xiàn)就能登錄上了 [root@docker-client ~]# docker login 172.16.60.213 #或者直接執(zhí)行"docker login -u admin -p kevin@BO123 172.16.60.213"命令登陸 Username: admin Password: Login Succeeded [root@docker-client ~]# 2)如果是在harbor本機(jī)登錄,出現(xiàn)上面的報(bào)錯(cuò): [root@harbor-node harbor]# docker login 172.16.60.213 Username: admin Password: Error response from daemon: Get https://172.16.60.213/v1/users/: dial tcp 172.16.60.213:443: connect: connection refused 解決辦法: 在/etc/docker/daemon.json 文件里添加"insecure-registries"配置 (第一行是之前添加的docker加速配置),注意兩行之間有一個(gè)","逗號(hào)隔開 [root@harbor-node harbor]# vim /etc/docker/daemon.json { "registry-mirrors": ["https://v5d7kh0f.mirror.aliyuncs.com"], "insecure-registries": ["172.16.60.213"] } 修改過后重啟docker, 然后重啟Harbor服務(wù) [root@harbor-node harbor]# systemctl restart docker [root@harbor-node harbor]# docker-compose stop [root@harbor-node harbor]# docker-compose start 然后再測(cè)試再harbor本機(jī)登錄 [root@harbor-node harbor]# docker login 172.16.60.213 Username: admin Password: WARNING! Your password will be stored unencrypted in /root/.docker/config.json. Configure a credential helper to remove this warning. See https://docs.docker.com/engine/reference/commandline/login/#credentials-store Login Succeeded [root@harbor-node harbor]# 登錄的賬號(hào)信息都保存到/root/.docker/config.json文件里了 [root@harbor-node harbor]# cat /root/.docker/config.json { "auths": { "172.16.60.213": { "auth": "YWRtaW46a2V2aW5AQk8xOTg3" } }, "HttpHeaders": { "User-Agent": "Docker-Client/18.09.6 (linux)" } 只要/root/.docker/config.json里的信息不刪除,后續(xù)再次登錄的時(shí)候,就不用輸入用戶名和密碼了 [root@harbor-node ~]# docker login 172.16.60.213 Authenticating with existing credentials... WARNING! Your password will be stored unencrypted in /root/.docker/config.json. Configure a credential helper to remove this warning. See https://docs.docker.com/engine/reference/commandline/login/#credentials-store Login Succeeded
注意事項(xiàng)總結(jié):
harbor支持http和https,但如果使用http的話,在拉取鏡像的時(shí)候,會(huì)拋出倉庫不受信任的異常。
需要在所有的docker客戶端的docker配置文件/etc/docker/daemon.json中添加如下配置:
{ "insecure-registries": ["https://*.*.*.*"] }
如果使用自簽名的https證書,仍然會(huì)提示證書不受信任的問題。需要將自簽名的ca證書發(fā)送到所有的docker客戶端的指定目錄。
關(guān)于使用自簽名證書配置harbor的具體過程可以參考: https://github.com/goharbor/harbor/blob/master/docs/configure_https.md
#鏡像打標(biāo)簽的命令 # docker tag 鏡像名:標(biāo)簽 私服地址/倉庫項(xiàng)目名/鏡像名:標(biāo)簽 #推送到私服的命令 #docker push 私服地址/倉庫項(xiàng)目名/鏡像名:標(biāo)簽 #從私服拉取鏡像的命令 #docker pull 私服地址/倉庫項(xiàng)目名/鏡像名:標(biāo)簽
首先在Harbor web界面里最好創(chuàng)建一個(gè)自己需要的"項(xiàng)目" (或者使用默認(rèn)的"library"項(xiàng)目),項(xiàng)目公開和私有:
- Public: 所有用戶對(duì)于公開項(xiàng)目都有讀權(quán)限,這種方式對(duì)于你想把一些倉庫分享給其他人的時(shí)候,是非常方便的.
- Private: 私有項(xiàng)目只能被有特定用戶權(quán)限的人去訪問。這種方式對(duì)于內(nèi)部團(tuán)隊(duì)來說共享也是比較方便的
比如創(chuàng)建一個(gè)公開項(xiàng)目"kevin_bo",點(diǎn)擊進(jìn)去可以看到推送鏡像的信息提示
然后就可以在Harbor服務(wù)器的終端命令行里進(jìn)行鏡像推送到Harbor倉庫的操作了:
在進(jìn)行harbor鏡像推送和拉取操作前,需要事先login登錄到harbor倉庫里,這樣才有harbor鏡像的推送和拉取的權(quán)限!! [root@harbor-node ~]# docker login 172.16.60.213 Authenticating with existing credentials... WARNING! Your password will be stored unencrypted in /root/.docker/config.json. Configure a credential helper to remove this warning. See https://docs.docker.com/engine/reference/commandline/login/#credentials-store Login Succeeded 先查看本機(jī)有哪些鏡像 [root@harbor-node ~]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE goharbor/redis-photon v1.8.0 66d7402d2770 10 days ago 103MB goharbor/harbor-registryctl v1.8.0 0ca3e2b624f5 10 days ago 96.2MB goharbor/registry-photon v2.7.1-patch-2819-v1.8.0 1e7d99ccba24 10 days ago 81.3MB goharbor/nginx-photon v1.8.0 4a4b48b32ae4 10 days ago 36MB goharbor/harbor-log v1.8.0 e718bdc405a3 10 days ago 81.5MB goharbor/harbor-jobservice v1.8.0 d47940dd883f 10 days ago 118MB goharbor/harbor-core v1.8.0 b07a1a4be17f 10 days ago 135MB goharbor/harbor-portal v1.8.0 76298a1ef089 10 days ago 42.9MB goharbor/harbor-db v1.8.0 d1e0b3df3e95 10 days ago 140MB goharbor/prepare v1.8.0 769ca785dab0 10 days ago 139MB 比如推送其中的goharbor/redis-photon:v1.8.0鏡像到Harbor倉庫的"kevin_bo"項(xiàng)目里 [root@harbor-node ~]# docker tag goharbor/redis-photon:v1.8.0 172.16.60.213/kevin_bo/redis-photon:v1.0 [root@harbor-node ~]# docker push 172.16.60.213/kevin_bo/redis-photon:v1.0 The push refers to repository [172.16.60.213/kevin_bo/redis-photon] 8864c4b9ac3d: Pushed 420b26399278: Pushed 4433bcd802e7: Pushed 268091c30a67: Pushed 23d9f72a5270: Pushed v1.0: digest: sha256:1e2ce8e6a852713d789c6315642d1483d1efdb4acee4699817810bef219ec93d size: 1366 查看本機(jī)的images,發(fā)現(xiàn)多了一個(gè)上面制作的鏡像,就是原來goharbor/redis-photon:v1.8.0的tag,可以選擇刪除 [root@harbor-node ~]# docker rmi 172.16.60.213/kevin_bo/redis-photon:v1.0 Untagged: 172.16.60.213/kevin_bo/redis-photon:v1.0 Untagged: 172.16.60.213/kevin_bo/redis-photon@sha256:1e2ce8e6a852713d789c6315642d1483d1efdb4acee4699817810bef219ec93d [root@harbor-node ~]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE goharbor/redis-photon v1.8.0 66d7402d2770 10 days ago 103MB goharbor/harbor-registryctl v1.8.0 0ca3e2b624f5 10 days ago 96.2MB goharbor/registry-photon v2.7.1-patch-2819-v1.8.0 1e7d99ccba24 10 days ago 81.3MB goharbor/nginx-photon v1.8.0 4a4b48b32ae4 10 days ago 36MB goharbor/harbor-log v1.8.0 e718bdc405a3 10 days ago 81.5MB goharbor/harbor-jobservice v1.8.0 d47940dd883f 10 days ago 118MB goharbor/harbor-core v1.8.0 b07a1a4be17f 10 days ago 135MB goharbor/harbor-portal v1.8.0 76298a1ef089 10 days ago 42.9MB goharbor/harbor-db v1.8.0 d1e0b3df3e95 10 days ago 140MB goharbor/prepare v1.8.0 769ca785dab0 10 days ago 139MB 同理,推送其他鏡像是同樣的操作,比如再推送goharbor/harbor-core:v1.8.0到harbor倉庫的"kevin_bo"項(xiàng)目里 [root@harbor-node ~]# docker tag goharbor/harbor-core:v1.8.0 172.16.60.213/kevin_bo/goharbor/harbor-core:v1.0 [root@harbor-node ~]# docker push 172.16.60.213/kevin_bo/goharbor/harbor-core:v1.0 The push refers to repository [172.16.60.213/kevin_bo/goharbor/harbor-core] 5385ffb8451e: Pushed 36e1cb2d6ffa: Pushed 452d238b3e48: Pushed af3a6f89469a: Pushed 05bc5efb1724: Pushed 23d9f72a5270: Mounted from kevin_bo/redis-photon v1.0: digest: sha256:7899f284617bb051180adf6c3aedd140a519d9092b8986dd9058d4dcec0d31de size: 1580 [root@harbor-node ~]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE goharbor/redis-photon v1.8.0 66d7402d2770 10 days ago 103MB goharbor/harbor-registryctl v1.8.0 0ca3e2b624f5 10 days ago 96.2MB goharbor/registry-photon v2.7.1-patch-2819-v1.8.0 1e7d99ccba24 10 days ago 81.3MB goharbor/nginx-photon v1.8.0 4a4b48b32ae4 10 days ago 36MB goharbor/harbor-log v1.8.0 e718bdc405a3 10 days ago 81.5MB goharbor/harbor-jobservice v1.8.0 d47940dd883f 10 days ago 118MB goharbor/harbor-core v1.8.0 b07a1a4be17f 10 days ago 135MB 172.16.60.213/kevin_bo/goharbor/harbor-core v1.0 b07a1a4be17f 10 days ago 135MB goharbor/harbor-portal v1.8.0 76298a1ef089 10 days ago 42.9MB goharbor/harbor-db v1.8.0 d1e0b3df3e95 10 days ago 140MB goharbor/prepare v1.8.0 769ca785dab0 10 days ago 139MB [root@harbor-node ~]# docker rmi 172.16.60.213/kevin_bo/goharbor/harbor-core:v1.0 Untagged: 172.16.60.213/kevin_bo/goharbor/harbor-core:v1.0 Untagged: 172.16.60.213/kevin_bo/goharbor/harbor-core@sha256:7899f284617bb051180adf6c3aedd140a519d9092b8986dd9058d4dcec0d31de [root@harbor-node ~]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE goharbor/redis-photon v1.8.0 66d7402d2770 10 days ago 103MB goharbor/harbor-registryctl v1.8.0 0ca3e2b624f5 10 days ago 96.2MB goharbor/registry-photon v2.7.1-patch-2819-v1.8.0 1e7d99ccba24 10 days ago 81.3MB goharbor/nginx-photon v1.8.0 4a4b48b32ae4 10 days ago 36MB goharbor/harbor-log v1.8.0 e718bdc405a3 10 days ago 81.5MB goharbor/harbor-jobservice v1.8.0 d47940dd883f 10 days ago 118MB goharbor/harbor-core v1.8.0 b07a1a4be17f 10 days ago 135MB goharbor/harbor-portal v1.8.0 76298a1ef089 10 days ago 42.9MB goharbor/harbor-db v1.8.0 d1e0b3df3e95 10 days ago 140MB goharbor/prepare v1.8.0 769ca785dab0 10 days ago 139MB
然后登錄到Harbor web 界面里,就可以看到"kevin_bo"項(xiàng)目里就有了上面推送的兩個(gè)鏡像了,點(diǎn)擊到對(duì)應(yīng)的鏡像了,還可以對(duì)鏡像進(jìn)行打標(biāo)簽,復(fù)制鏡像等操作:
比如將"kevin_bo"項(xiàng)目里上面的kevin_bo/goharbor/harbor-core鏡像復(fù)制到"library"項(xiàng)目里
然后到"library"項(xiàng)目里就能看到上面從"kevin_bo"項(xiàng)目里復(fù)制過來的鏡像了
注意: harbor私倉的相關(guān)容器映射到主機(jī)的volumes數(shù)據(jù)卷的空間要有保證,最好是單獨(dú)的分區(qū)空間。
上面測(cè)試harbor容器通過volumes映射到主機(jī)的目錄是/data, 可以到這里查看harbor推送的鏡像:
[root@harbor-node repositories]# pwd /data/registry/docker/registry/v2/repositories 可以查看兩個(gè)"項(xiàng)目" [root@harbor-node repositories]# ll total 0 drwxr-xr-x 4 10000 10000 42 May 27 14:01 kevin_bo drwxr-xr-x 3 10000 10000 27 May 27 14:08 library [root@harbor-node repositories]# ll kevin_bo/ total 0 drwxr-xr-x 3 10000 10000 25 May 27 14:01 goharbor drwxr-xr-x 5 10000 10000 55 May 27 13:58 redis-photon [root@harbor-node repositories]# ll library/ total 0 drwxr-xr-x 3 10000 10000 21 May 27 14:08 172.16.60.213 [root@harbor-node repositories]# ll library/172.16.60.213/ total 0 drwxr-xr-x 3 10000 10000 25 May 27 14:08 library
========測(cè)試下在harbor客戶端下載harbor倉庫里的鏡像=======
[root@client ~]# docker login 172.16.60.213 Username: admin Password: Login Succeeded [root@client ~]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE 從harbor倉庫拉取鏡像 [root@client ~]# docker pull 172.16.60.213/kevin_bo/goharbor/harbor-core:v1.0 Trying to pull repository 172.16.60.213/kevin_bo/goharbor/harbor-core ... v1.0: Pulling from 172.16.60.213/kevin_bo/goharbor/harbor-core 4e360eca2e60: Pull complete c066267eb2b9: Pull complete 932afda2a169: Pull complete 7ed16fb7e79a: Pull complete d09137d80617: Pull complete 588769341947: Pull complete Digest: sha256:7899f284617bb051180adf6c3aedd140a519d9092b8986dd9058d4dcec0d31de Status: Downloaded newer image for 172.16.60.213/kevin_bo/goharbor/harbor-core:v1.0 [root@client ~]# docker pull 172.16.60.213/kevin_bo/redis-photon:v1.0 Trying to pull repository 172.16.60.213/kevin_bo/redis-photon ... v1.0: Pulling from 172.16.60.213/kevin_bo/redis-photon 4e360eca2e60: Already exists b08cc3be5c43: Pull complete a750a309c85d: Pull complete 49b2d8335a1a: Pull complete 31e8f89dc042: Pull complete Digest: sha256:1e2ce8e6a852713d789c6315642d1483d1efdb4acee4699817810bef219ec93d Status: Downloaded newer image for 172.16.60.213/kevin_bo/redis-photon:v1.0 [root@client ~]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE 172.16.60.213/kevin_bo/redis-photon v1.0 66d7402d2770 10 days ago 103 MB 172.16.60.213/kevin_bo/goharbor/harbor-core v1.0 b07a1a4be17f 10 days ago 135 MB
可以在登錄Harbor web界面之后,修改相關(guān)用戶的密碼。在不同用戶賬號(hào)下創(chuàng)建項(xiàng)目,以及推送和拉取harbor鏡像操作。
通過上面可知,harbor默認(rèn)安裝后采用的是http方式,后面使用的時(shí)候可能會(huì)發(fā)現(xiàn)很多不方面。因?yàn)镈ocker客戶端登錄harbor進(jìn)行鏡像推送或拉取時(shí)默認(rèn)是https方式!所以http方式下,需要在每一臺(tái)harbor客戶端機(jī)器上都要設(shè)置"insecure-registries", 感覺很麻煩!所以最好還是將harbor默認(rèn)的http方式改為https方式!另外,從安全角度考慮,容器的倉庫在生產(chǎn)環(huán)境中往往也是需要被設(shè)定為https的方式,而harbor將這些證書的創(chuàng)建和設(shè)定都進(jìn)行了簡(jiǎn)單的集成,下面來看一下在harbor下如何使用https的方式。配置記錄如下:
在創(chuàng)建證書之前,為了方面驗(yàn)證,需要將前面在客戶端機(jī)器上/etc/docker/daemon.json文件里添加"insecure-registries"配置刪除 [root@docker-client ~]# vim /etc/docker/daemon.json {} [root@docker-client ~]# rm -rf /root/.docker 重啟docker服務(wù) [root@docker-client ~]# systemctl restart docker 將harbor部署機(jī)自身的/etc/docker/daemon.json文件里添加"insecure-registries"配置也刪除 [root@harbor-node ~]# vim /etc/docker/daemon.json { "registry-mirrors": ["https://v5d7kh0f.mirror.aliyuncs.com"] } [root@harbor-node ~]# rm -rf /root/.docker 然后重啟docker和docker-compose [root@harbor-node ~]# systemctl restart docker [root@harbor-node ~]# docker-compose down -t [root@harbor-node ~]# docker-compose up -d 1)創(chuàng)建CA [root@harbor-node harbor]# pwd /root/harbor [root@harbor-node harbor]# mkdir ssl [root@harbor-node harbor]# cd ssl/ [root@harbor-node ssl]# pwd /root/harbor/ssl [root@harbor-node ssl]# openssl req -newkey rsa:4096 -nodes -sha256 -keyout ca.key -x509 -days 365 -out ca.crt Generating a 4096 bit RSA private key ......................................++ ...................................................................................................................................................................................................................................................................++ writing new private key to 'ca.key' ----- You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----- Country Name (2 letter code) [XX]:CN State or Province Name (full name) []:beijing Locality Name (eg, city) [Default City]:beijing Organization Name (eg, company) [Default Company Ltd]:DevOps Organizational Unit Name (eg, section) []:Tec Common Name (eg, your name or your server's hostname) []:172.16.60.213 Email Address []:wangshibo@kevin.com 2) 創(chuàng)建證書請(qǐng)求文件csr [root@harbor-node ssl]# openssl req -newkey rsa:4096 -nodes -sha256 -keyout 172.16.60.213.key -out 172.16.60.213.csr Generating a 4096 bit RSA private key .++ ..........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................++ writing new private key to '172.16.60.213.key' ----- You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----- Country Name (2 letter code) [XX]:CN State or Province Name (full name) []:beijing Locality Name (eg, city) [Default City]:beijing Organization Name (eg, company) [Default Company Ltd]:DevOps Organizational Unit Name (eg, section) []:Tec Common Name (eg, your name or your server's hostname) []:172.16.60.213 Email Address []:wangshibo@kevin.com Please enter the following 'extra' attributes to be sent with your certificate request A challenge password []:123456 An optional company name []:DevOps 3) 創(chuàng)建證書 [root@harbor-node ssl]# echo subjectAltName = IP:172.16.60.213 > extfile.cnf [root@harbor-node ssl]# openssl x509 -req -days 365 -in 172.16.60.213.csr -CA ca.crt -CAkey ca.key -CAcreateserial -extfile extfile.cnf -out 172.16.60.213.crt Signature ok subject=/C=CN/ST=beijing/L=beijing/O=DevOps/OU=Tec/CN=172.16.60.213/emailAddress=wangshibo@kevin.com Getting CA Private Key 4) 設(shè)定證書 & 修改 查看證書所在路徑, 后面將harbor.yaml文件中的路徑也同樣設(shè)定 [root@harbor-node ssl]# pwd /root/harbor/ssl [root@harbor-node ssl]# ll total 28 -rw-r--r-- 1 root root 2033 May 28 01:16 172.16.60.213.crt -rw-r--r-- 1 root root 1809 May 28 01:15 172.16.60.213.csr -rw-r--r-- 1 root root 3272 May 28 01:15 172.16.60.213.key -rw-r--r-- 1 root root 2114 May 28 01:13 ca.crt -rw-r--r-- 1 root root 3268 May 28 01:13 ca.key -rw-r--r-- 1 root root 17 May 28 01:16 ca.srl -rw-r--r-- 1 root root 34 May 28 01:16 extfile.cnf 5) 修改harbor.yaml文件 先關(guān)閉docker-compose [root@harbor-node harbor]# pwd /root/harbor [root@harbor-node harbor]# docker-compose down -v Stopping nginx ... done Stopping harbor-jobservice ... done Stopping harbor-portal ... done Stopping harbor-core ... done Stopping harbor-db ... done Stopping registryctl ... done Stopping redis ... done Stopping registry ... done Stopping harbor-log ... done Removing nginx ... done Removing harbor-jobservice ... done Removing harbor-portal ... done Removing harbor-core ... done Removing harbor-db ... done Removing registryctl ... done Removing redis ... done Removing registry ... done Removing harbor-log ... done Removing network harbor_harbor [root@harbor-node harbor]# docker-compose ps Name Command State Ports ------------------------------ [root@harbor-node harbor]# vim harbor.yml (可以將80端口的配置注釋了,直接使用443端口配置。docker login https://ip 登錄即可) ................. ................. # http related config http: # port for http, default is 80. If https enabled, this port will redirect to https port port: 80 # https related config https: # # https port for harbor, default is 443 port: 443 # # The path of cert and key files for nginx certificate: /root/harbor/ssl/172.16.60.213.crt private_key: /root/harbor/ssl/172.16.60.213.key ................. ................. ================================================================================================================ 特別注意: 上面harbor.yaml文件中修改的配置格式一定要正確!"https"要頂格寫,"port:443" 和 "certificate"、"private_key"保持縮進(jìn)一致! 否則在下面執(zhí)行"./prepare"更新命令時(shí),會(huì)報(bào)錯(cuò): [root@harbor-node harbor]# ./prepare .......... File "/usr/lib/python3.6/site-packages/yaml/composer.py", line 84, in compose_node node = self.compose_mapping_node(anchor) File "/usr/lib/python3.6/site-packages/yaml/composer.py", line 127, in compose_mapping_node while not self.check_event(MappingEndEvent): File "/usr/lib/python3.6/site-packages/yaml/parser.py", line 98, in check_event self.current_event = self.state() File "/usr/lib/python3.6/site-packages/yaml/parser.py", line 439, in parse_block_mapping_key "expected <block end>, but found %r" % token.id, token.start_mark) yaml.parser.ParserError: while parsing a block mapping in "/input/harbor.yml", line 15, column 4 expected <block end>, but found '<block mapping start>' in "/input/harbor.yml", line 17, column 5 上面的報(bào)錯(cuò),就是由于harbor.yaml文件配置格式不正確導(dǎo)致的?。。?! ================================================================================================================ 接著執(zhí)行prepare腳本,將harbor修改的配置更新到 docker-compose.yml 文件 [root@harbor-node harbor]# ./prepare prepare base dir is set to /root/harbor Clearing the configuration file: /config/log/logrotate.conf Clearing the configuration file: /config/nginx/nginx.conf Clearing the configuration file: /config/core/env Clearing the configuration file: /config/core/app.conf Clearing the configuration file: /config/registry/config.yml Clearing the configuration file: /config/registry/root.crt Clearing the configuration file: /config/registryctl/env Clearing the configuration file: /config/registryctl/config.yml Clearing the configuration file: /config/db/env Clearing the configuration file: /config/jobservice/env Clearing the configuration file: /config/jobservice/config.yml Generated configuration file: /config/log/logrotate.conf Generated configuration file: /config/nginx/nginx.conf Generated configuration file: /config/core/env Generated configuration file: /config/core/app.conf Generated configuration file: /config/registry/config.yml Generated configuration file: /config/registryctl/env Generated configuration file: /config/db/env Generated configuration file: /config/jobservice/env Generated configuration file: /config/jobservice/config.yml loaded secret from file: /secret/keys/secretkey Generated configuration file: /compose_location/docker-compose.yml Clean up the input dir 查看一下docker-compose.yml文件,發(fā)現(xiàn)已經(jīng)將新配置的443端口的https信息更新到docker-compose.yml文件里了 如下80端口和443端口都配置了,所以harbor訪問時(shí)是http強(qiáng)轉(zhuǎn)到https的 [root@harbor-node harbor]# cat docker-compose.yml |grep 443 -C3 dns_search: . ports: - 80:80 - 443:443 depends_on: - postgresql - registry 重啟docker-compose [root@harbor-node harbor]# docker-compose up -d Creating network "harbor_harbor" with the default driver Creating harbor-log ... done Creating registry ... done Creating harbor-db ... done Creating registryctl ... done Creating redis ... done Creating harbor-core ... done Creating harbor-jobservice ... done Creating harbor-portal ... done Creating nginx ... done [root@harbor-node harbor]# docker-compose ps Name Command State Ports ------------------------------------------------------------------------------------------------------------ harbor-core /harbor/start.sh Up (healthy) harbor-db /entrypoint.sh postgres Up (healthy) 5432/tcp harbor-jobservice /harbor/start.sh Up harbor-log /bin/sh -c /usr/local/bin/ ... Up (healthy) 127.0.0.1:1514->10514/tcp harbor-portal nginx -g daemon off; Up (healthy) 80/tcp nginx nginx -g daemon off; Up (healthy) 0.0.0.0:443->443/tcp, 0.0.0.0:80->80/tcp redis docker-entrypoint.sh redis ... Up 6379/tcp registry /entrypoint.sh /etc/regist ... Up (healthy) 5000/tcp registryctl /harbor/start.sh Up (healthy) 在harbor部署機(jī)本機(jī)確認(rèn)login登陸 (使用80端口或443端口都可以,自動(dòng)跳轉(zhuǎn)的) [root@harbor-node harbor]# docker login -u admin -p kevin@Bo1987 172.16.60.213 WARNING! Using --password via the CLI is insecure. Use --password-stdin. Error response from daemon: Get https://172.16.60.213/v2/: x509: certificate signed by unknown authority [root@harbor-node harbor]# docker login -u admin -p kevin@Bo1987 172.16.60.213:443 WARNING! Using --password via the CLI is insecure. Use --password-stdin. Error response from daemon: Get https://172.16.60.213:443/v2/: x509: certificate signed by unknown authority 以上出現(xiàn)報(bào)錯(cuò),解決辦法: 此種情況多發(fā)生在自簽名的證書,報(bào)錯(cuò)含義是簽發(fā)證書機(jī)構(gòu)未經(jīng)認(rèn)證,無法識(shí)別。 解決辦法: [root@harbor-node harbor]# chmod 644 /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem [root@harbor-node harbor]# cat /root/harbor/ssl/172.16.60.213.crt >> /etc/pki/tls/certs/ca-bundle.crt [root@harbor-node harbor]# chmod 444 /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 由于證書是docker的daemon需要用到的,所以需要重啟docker服務(wù),進(jìn)而也要重啟docker-compose [root@harbor-node harbor]# systemctl restart docker [root@harbor-node harbor]# docker-compose down -v [root@harbor-node harbor]# docker-compose up -d 然后再次嘗試在harbor本機(jī)登陸, 發(fā)現(xiàn)就可以正常登陸了??! [root@harbor-node harbor]# docker login -u admin -p kevin@Bo1987 172.16.60.213 WARNING! Using --password via the CLI is insecure. Use --password-stdin. WARNING! Your password will be stored unencrypted in /root/.docker/config.json. Configure a credential helper to remove this warning. See https://docs.docker.com/engine/reference/commandline/login/#credentials-store Login Succeeded 登陸的權(quán)限信息保存到/root/.docker/config.json 文件里了,只要這個(gè)文件不刪除,下次就可以不需要輸入用戶名和密碼直接登陸了! [root@harbor-node harbor]# cat /root/.docker/config.json { "auths": { "172.16.60.213": { "auth": "YWRtaW46a2V2aW5AQk9CTzEyMw==" } }, "HttpHeaders": { "User-Agent": "Docker-Client/18.09.6 (linux)" } } [root@harbor-node harbor]# docker login 172.16.60.213 Authenticating with existing credentials... WARNING! Your password will be stored unencrypted in /root/.docker/config.json. Configure a credential helper to remove this warning. See https://docs.docker.com/engine/reference/commandline/login/#credentials-store Login Succeeded 上面是使用80端口登陸的,后面加上443端口也是可以登陸的 [root@harbor-node harbor]# docker login 172.16.60.213:443 Username: admin Password: WARNING! Your password will be stored unencrypted in /root/.docker/config.json. Configure a credential helper to remove this warning. See https://docs.docker.com/engine/reference/commandline/login/#credentials-store Login Succeeded [root@harbor-node harbor]# docker login 172.16.60.213:443 Authenticating with existing credentials... WARNING! Your password will be stored unencrypted in /root/.docker/config.json. Configure a credential helper to remove this warning. See https://docs.docker.com/engine/reference/commandline/login/#credentials-store Login Succeeded [root@harbor-node ssl]# docker login -u admin -p kevin@Bo1987 172.16.60.213:443 WARNING! Using --password via the CLI is insecure. Use --password-stdin. WARNING! Your password will be stored unencrypted in /root/.docker/config.json. Configure a credential helper to remove this warning. See https://docs.docker.com/engine/reference/commandline/login/#credentials-store Login Succeeded ========================================================================================== 上面是在harbor本機(jī)嘗試的登陸,現(xiàn)在在遠(yuǎn)程客戶機(jī)上(這里客戶機(jī)為172.16.60.214)測(cè)試harbor登陸: 首先很重要的一步,這一步極其關(guān)鍵!一定不要忘記操作??! 就是需要將harbor服務(wù)端生成的CA證書拷貝到每個(gè)遠(yuǎn)程客戶機(jī)的"/etc/docker/certs.d/harbor服務(wù)器的域名或ip/" 目錄下 [root@client ~]# mkdir /etc/docker/certs.d/172.16.60.213/ 接著在harbor服務(wù)器將CA證書拷貝過來 [root@harbor-node ssl]# rsync -e "ssh -p22" -avpgolr ./* root@172.16.60.214:/etc/docker/certs.d/172.16.60.213/ 然后在客戶機(jī)上查看是否拷貝過來了harbor服務(wù)端的CA證書 [root@client 172.16.60.213]# pwd /etc/docker/certs.d/172.16.60.213 [root@client 172.16.60.213]# ls 172.16.60.213.crt 172.16.60.213.csr 172.16.60.213.key ca.crt ca.key ca.srl extfile.cnf 進(jìn)行同樣的授權(quán)操作, [root@client 172.16.60.213]# chmod 644 /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem [root@client 172.16.60.213]# cat /etc/docker/certs.d/172.16.60.213/172.16.60.213.crt >> /etc/pki/tls/certs/ca-bundle.crt [root@client 172.16.60.213]# chmod 444 /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 重啟docker服務(wù) [root@client 172.16.60.213]# systemctl restart docker 最后進(jìn)行harbor登陸,就可以正常登陸了! [root@client 172.16.60.213]# docker login -u admin -p kevin@Bo1987 172.16.60.213:443 Login Succeeded [root@client 172.16.60.213]# cat /root/.docker/config.json { "auths": { "172.16.60.213:443": { "auth": "YWRtaW46a2V2aW5AQk9CTzEyMw==" } } } 上面是使用443端口登陸harbor倉庫是正常的,如果此時(shí)使用80端口登陸,則出現(xiàn)如下報(bào)錯(cuò): [root@client 172.16.60.213]# docker login -u admin -p kevin@Bo1987 172.16.60.213 Error response from daemon: Missing client certificate 172.16.60.213.cert for key 172.16.60.213.key 所以在客戶端就使用443端口來登陸harbor倉庫了!
[[ 需要注意 ]] 如果Harbor里創(chuàng)建了多個(gè)賬號(hào),客戶端使用A賬號(hào)登錄harbor后,docker pull下載的鏡像是在B賬號(hào)的項(xiàng)目里面的,并且該項(xiàng)目是私有的,那么需要先將A賬號(hào)添加為該項(xiàng)目的成員后才能正常docker pull下載,否則會(huì)出現(xiàn)報(bào)錯(cuò):
denied: requested access to the resource is denied
單機(jī)部署harbor顯然無法滿足在生產(chǎn)中需求,必須要保證應(yīng)用的高可用性。
目前有兩種主流的方案來解決Harbor高可用問題:
- 雙主復(fù)制
- 多harbor實(shí)例共享后端存儲(chǔ)
1. Harbor雙主復(fù)制
- 主從同步
harbor官方默認(rèn)提供主從復(fù)制的方案來解決鏡像同步問題,通過復(fù)制方式,可以實(shí)時(shí)將測(cè)試環(huán)境harbor倉庫的鏡像同步到生產(chǎn)環(huán)境harbor,類似于如下流程:
在實(shí)際生產(chǎn)運(yùn)維的中,往往需要把鏡像發(fā)布到幾十或上百臺(tái)集群節(jié)點(diǎn)上。這時(shí),單個(gè)Registry已經(jīng)無法滿足大量節(jié)點(diǎn)的下載需求,因此要配置多個(gè)Registry實(shí)例做負(fù)載均衡。手工維護(hù)多個(gè)Registry實(shí)例上的鏡像,將是十分繁瑣的事情。Harbor可以支持一主多從的鏡像發(fā)布模式,可以解決大規(guī)模鏡像發(fā)布的難題。
只要往一臺(tái)Registry上發(fā)布,鏡像就像"仙女散花"般地同步到多個(gè)Registry中,高效可靠。如果是地域分布較廣的集群,還可以采用層次型發(fā)布方式,如從集團(tuán)總部同步到省公司,從省公司再同步到市公司。
然而單靠主從同步方式仍然解決不了harbor主節(jié)點(diǎn)的單點(diǎn)問題。
- 雙主復(fù)制說明
所謂雙主復(fù)制其實(shí)就是復(fù)用主從同步實(shí)現(xiàn)兩個(gè)harbor節(jié)點(diǎn)之間的雙向同步,來保證數(shù)據(jù)的一致性,然后在兩臺(tái)harbor前端配置一個(gè)負(fù)載均衡器將進(jìn)來的請(qǐng)求分流到不同的實(shí)例中去,只要有一個(gè)實(shí)例中有了新的鏡像,就是自動(dòng)的同步復(fù)制到另外的的實(shí)例中去,這樣實(shí)現(xiàn)了負(fù)載均衡,也避免了單點(diǎn)故障,在一定程度上實(shí)現(xiàn)了Harbor的高可用性。可以使用下面方案:Nginx+Keepalive+Harbor,VIP可以在LB上實(shí)現(xiàn)漂移。(或者VIP直接在Harbor之間漂移)。
創(chuàng)建harbor主主復(fù)制,可以在harbor的web界面里創(chuàng)建相互之間鏡像同步關(guān)系,同步關(guān)系可以選擇相同用戶或不同用戶之間。這樣就保證了harbor雙機(jī)熱備關(guān)系。
1) "系統(tǒng)管理"->"倉庫管理"->"新建目標(biāo)", 填寫對(duì)端harbor信息
2) "系統(tǒng)管理"->"同步管理"->"新建規(guī)則", 規(guī)則里會(huì)引用目的Registry,也就是上面一步創(chuàng)建的目標(biāo)。同步模式有Push-based,Pull-based;觸發(fā)模式有自動(dòng)和定時(shí)。
不過這個(gè)方案有一個(gè)問題:就是有可能兩個(gè)Harbor實(shí)例中的數(shù)據(jù)不一致。假設(shè)如果一個(gè)實(shí)例A掛掉了,這個(gè)時(shí)候有新的鏡像進(jìn)來,那么新的鏡像就會(huì)在另外一個(gè)實(shí)例B中,后面即使恢復(fù)了掛掉的A實(shí)例,Harbor實(shí)例B也不會(huì)自動(dòng)去同步鏡像,這樣只能手動(dòng)的先關(guān)掉Harbor實(shí)例B的復(fù)制策略,然后再開啟復(fù)制策略,才能讓實(shí)例B數(shù)據(jù)同步,讓兩個(gè)實(shí)例的數(shù)據(jù)一致。所以,在實(shí)際生產(chǎn)使用中,主從復(fù)制十分的不靠譜。
共享后端存儲(chǔ)算是一種比較標(biāo)準(zhǔn)的方案,就是多個(gè)Harbor實(shí)例共享同一個(gè)后端存儲(chǔ),任何一個(gè)實(shí)例持久化到存儲(chǔ)的鏡像,都可被其他實(shí)例中讀取。通過前置LB進(jìn)來的請(qǐng)求,可以分流到不同的實(shí)例中去處理,這樣就實(shí)現(xiàn)了負(fù)載均衡,也避免了單點(diǎn)故障。
這個(gè)方案在實(shí)際生產(chǎn)環(huán)境中部署需要考慮三個(gè)問題:
1. 共享存儲(chǔ)的選取,Harbor的后端存儲(chǔ)目前支持AWS S3、Openstack Swift、Ceph等,[在實(shí)驗(yàn)環(huán)境里,可以直接使用nfs]
2. Session在不同的實(shí)例上共享,這個(gè)現(xiàn)在其實(shí)已經(jīng)不是問題了,在最新的harbor中,默認(rèn)session會(huì)存放在redis中,我們只需要將redis獨(dú)立出來即可??梢酝ㄟ^redis sentinel或者redis cluster等方式來保證redis的可用性。[在實(shí)驗(yàn)環(huán)境里,仍然使用單臺(tái)redis]
3. Harbor多實(shí)例數(shù)據(jù)庫問題,這個(gè)只需要將harbor中的數(shù)據(jù)庫拆出來獨(dú)立部署即可。讓多實(shí)例共用一個(gè)外部數(shù)據(jù)庫,外部數(shù)據(jù)庫可以通過Mysql 高可用方案保證高可用性。
到此,相信大家對(duì)“Docker私有倉庫Harbor怎么部署”有了更深的了解,不妨來實(shí)際操作一番吧!這里是億速云網(wǎng)站,更多相關(guān)內(nèi)容可以進(jìn)入相關(guān)頻道進(jìn)行查詢,關(guān)注我們,繼續(xù)學(xué)習(xí)!
免責(zé)聲明:本站發(fā)布的內(nèi)容(圖片、視頻和文字)以原創(chuàng)、轉(zhuǎn)載和分享為主,文章觀點(diǎn)不代表本網(wǎng)站立場(chǎng),如果涉及侵權(quán)請(qǐng)聯(lián)系站長(zhǎng)郵箱:is@yisu.com進(jìn)行舉報(bào),并提供相關(guān)證據(jù),一經(jīng)查實(shí),將立刻刪除涉嫌侵權(quán)內(nèi)容。