oracle 19c rac install 之grid install

一 前置檢查：
安裝最低要求：
Red Hat Enterprise Linux 7.4: 3.10.0-693.5.2.0.1.el7.x86_64 or later
At least 8 GB RAM for Oracle Grid Infrastructure installations. （其實(shí)4G也可以安裝完成）
SWAP：4G
安裝目錄：grid 10G,介質(zhì)安裝完成6.4G
參考:grid install guide
https://docs.oracle.com/en/database/oracle/oracle-database/19/cwlin/server-hardware-checklist-for-oracle-grid-infrastructure.html#GUID-D964EF96-9C9C-4173-A919-F3CAA72180B1
這里環(huán)境概述：
IP：和11G準(zhǔn)備一樣，5個(gè)公網(wǎng)，2個(gè)私網(wǎng)；
共享存儲(chǔ)：5GB；
操作系統(tǒng)：RHEL7.6 x64
介質(zhì)準(zhǔn)備：V981624-01.zip，V981627-01.zip，V981623-01.zip三個(gè)壓縮文件，比18C小一些；只有for exadata版 本，在edelivery.oracle.com下載，公版沒(méi)出來(lái)，不過(guò)裝在linux應(yīng)當(dāng)是沒(méi)有問(wèn)題；介質(zhì)變化：
1 軟件解壓目錄即安裝目錄
2 自帶JDK 1.8/TFA
3 logfile默認(rèn)由50調(diào)到了200M

二 網(wǎng)絡(luò)配置
OS采用最小安裝，安裝過(guò)程略，安裝完成后先啟用臨時(shí)網(wǎng)絡(luò)：

nmcli con modify "System eth0" ipv4.addresses 192.168.56.102/24 
nmcli con up "System eth0" 
ip addr show

設(shè)置主機(jī)名：
hostnamectl set-hostname rhel6
由于rhel7的網(wǎng)卡名稱命名變化,如需顯示eth0,eth2,需在RHEL 7修改網(wǎng)卡參數(shù)：
1、修改/etc/sysconfig/grub或者/etc/default/grub，它們是一個(gè)文件，改誰(shuí)都可以，需要添加參數(shù)，給GRUB_CMDLINE_LINUX參數(shù)中增加
net.ifnames=0 biosdevname=0
然后：
grub2-mkconfig -o /boot/grub2/grub.cfg
2、配置網(wǎng)卡

mv /etc/sysconfig/network-scripts/ifcfg-enp0s3 /etc/sysconfig/network-scripts/ifcfg-eth0
vi /etc/sysconfig/network-scripts/ifcfg-eth0
修改：
device=eth0
name=eth0

*3、非必須，更改/etc/udev/ruled.d/90-eno-fix.rules，將其中的NAME="enp0s3"改為eth0
最后重啟主機(jī)生效：
#禁用dhcp 169.254路由表

echo "NOZEROCONF=yes">>/etc/sysconfig/network
/etc/init.d/network restart

#RHEL 7禁用防火墻

 systemctl stop firewalld
systemctl mask firewalld

三 配置共享磁盤(pán)屬性

vi /etc/rc.d/rc.local 
/usr/bin/raw /dev/raw/raw1 /dev/sdb
/usr/bin/chown grid:asmadmin /dev/raw/raw1
/usr/bin/chmod 660 /dev/raw/raw1
chmod +x /etc/rc.d/rc.local

*四 建立用戶
如果先安裝oracle的yum包，會(huì)自動(dòng)建立

$ id oracle
uid=54321(oracle) gid=54321(oinstall) groups=54321(oinstall),54322(dba), 
54323(oper),54324(backupdba),54325(dgdba),54326(kmdba),54327(asmdba),54330(racdba)
$ id grid
uid=54331(grid) gid=54321(oinstall) groups=54321(oinstall),54322(dba),
54327(asmdba),54328(asmoper),54329(asmadmin),54330(racdba)

這種用戶

groupadd -g 501 oinstall
groupadd -g 502 dba
groupadd -g 504 asmadmin
groupadd -g 506 asmdba
groupadd -g 507 asmoper
useradd -u 501 -g oinstall -G asmadmin,asmdba,asmoper grid
useradd -u 502 -g oinstall -G dba,asmdba oracle
echo 'oracle123' |passwd oracle --stdin;
echo 'oracle123' |passwd grid --stdin;

--配置ssh信任關(guān)系
./oui/prov/resources/scripts/sshUserSetup.sh -user grid -hosts "rhel6 rhel7" -noPromptPassphrase
或者
ssh-keygen生成用戶密鑰,配置互信；

五 配置yum 軟件包
cd /etc/yum.repos.d/
mkdir bak
mv Centos ./bak
直接：
wget http://yum.oracle.com/public-yum-ol7.repo
修改其中的ol7_latest
或者新建一個(gè)repo文件：

[ol7_latest]
name=Oracle Linux $releasever Latest ($basearch)
baseurl=https://yum.oracle.com/repo/OracleLinux/OL7/latest/$basearch/
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
gpgcheck=0
enabled=1

由于最小安裝，需補(bǔ)充包:

yum install oracle-database-preinstall-18c
yum -y install gcc*
yun -y install xhost+

當(dāng)然也可以用傳統(tǒng)的掛載鏡像來(lái)做yum源

mount -o loop /tmp/rhel-server-6.6-x86_64-dvd.iso /media
vi /etc/yum.respos.d/dvd.repo
[c6-media]
name=CentOS-$releasever - Media
baseurl=file:///media
gpgcheck=0
enabled=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6

wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-6.repo
rpm -q --qf '%{NAME}-%{VERSION}-%{RELEASE} (%{ARCH})\n' rpm -q --qf '%{NAME}-%{VERSION}-%{RELEASE} (%{ARCH})\n' bc \
binutils \
compat-libcap1 \
compat-libstdc++ \
dtrace-modules \
dtrace-modules-headers \
dtrace-modules-provider-headers \
dtrace-utils \
elfutils-libelf \
elfutils-libelf-devel \
fontconfig-devel \
glibc \
glibc-devel \
ksh \
libaio \
libaio-devel \
libdtrace-ctf-devel \
libX11 \
libXau \
libXi \
libXtst \
libXrender \
libXrender-devel \
libgcc \
librdmacm-devel \
libstdc++ \
libstdc++-devel \
libxcb 

rpm -q --qf '%{NAME}-%{VERSION}-%{RELEASE} (%{ARCH})\n' binutils compat-libstdc++-33 elfutils-libelf elfutils-libelf-devel gcc gcc-c++ glibc glibc-common glibc-devel glibc-headers ksh libaio libaio-devel libgcc libstdc++ libstdc++-devel make sysstat unixODBC unixODBC-devel|grep -i "not installed"|awk '{print $2}'|xargs yum install -y

如果為RHEL6得升級(jí)glibc,libstdc++庫(kù)到2.14/4.5以下版本；
檢查版本:```
ldd --version
[grid@rhel6 grid]$ strings /usr/lib64/libstdc++.so.6|grep GLIBCXX
GLIBCXX_3.4
GLIBCXX_3.4.1
GLIBCXX_3.4.2
GLIBCXX_3.4.3
GLIBCXX_3.4.4
GLIBCXX_3.4.5
GLIBCXX_3.4.6
GLIBCXX_3.4.7
GLIBCXX_3.4.8
GLIBCXX_3.4.9
GLIBCXX_3.4.10
GLIBCXX_3.4.11
GLIBCXX_3.4.12
GLIBCXX_3.4.13
GLIBCXX_FORCE_NEW
GLIBCXX_DEBUG_MESSAGE_LENGTH

**六 配置hosts**

vi /etc/hosts
127.0.0.1 localhost
#eth0 - PUBLIC
192.168.56.101 rhel7
192.168.56.102 rhel6
#VIP
192.168.56.103 rhel7-vip
192.168.56.104 rhel6-vip
#eth2 - PRIVATE
10.0.2.15 rhel7-pvt
10.0.2.16 rhel6-pvt
192.168.56.105 rac-scan

**七 配置 參數(shù)**

RHEL7 yum install后可以忽略

vi /etc/sysctl.conf
kernel.shmall = 4294967296
kernel.shmmni =4096
kernel.sem = 250 32000 100 128
fs.file-max = 6815744
net.ipv4.ip_local_port_range = 9000 65000
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
fs.aio-max-nr = 1048576
kernel.panic_on_oops=1

#sysctl -p

#調(diào)整shm:

tmpfs /dev/shm tmpfs defaults,size=3000M 0 0
mount -o remount /dev/shm

修改limit參數(shù)：
/etc/security/limits.conf

grid soft nproc 4096
grid hard nproc 16384
grid soft nofile 4096
grid hard nofile 65536
oracle soft nproc 4096
oracle hard nproc 16384
oracle soft nofile 4096
oracle hard nofile 65536

RHEL6注意可能不生效的情況需修改：
`/etc/security/limits.d/90-nproc.conf`
來(lái)源詳細(xì)參見(jiàn)：
http://blog.yufeng.info/archives/2568

vi /etc/pam.d/login
session required pam_limits.so

vi /etc/profile
if [ $USER = "oracle" ] || [ $USER = "grid" ]; then
if [ $SHELL = "/bin/ksh" ]; then
ulimit -p 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
umask 022
fi

#關(guān)閉selinux
setenforce 0
vi /etc/sysconfig/selinux，修改target參數(shù)為disabled

**8 配置環(huán)境變量**

export ORACLE_HOME=/data/grid
export ORACLE_BASE=/u01/grid
export ORACLE_SID=+ASM2
export DISPLAY=192.168.56.1:0.0

PATH=$PATH:$HOME/.local/bin:$HOME/bin:$ORACLE_HOME/bin

**9 安裝**
執(zhí)行安裝前檢查
` ./runcluvfy.sh stage -pre crsinst -n rhel6,rhel7 -verbose`

>  Verifying DNS/NIS name service ...FAILED
> PRVG-2058 : The 'hosts' entries in the existing "/etc/nsswitch.conf" files are
> inconsistent.
> "hosts" entry was found as "hosts: files dns myhostname" on nodes: rhel7
> "hosts" entry was found as "hosts: files dns" on nodes: rhel6
>  
> Verifying Daemon "avahi-daemon" not configured and running ...FAILED
> Verifying RPM Package Manager database ...INFORMATION
> PRVG-11250 : The check "RPM Package Manager database" was not performed because it needs 'root' user privileges.

一個(gè)是DNS解析文件的順序不一致，一個(gè)是avahi-daemon沒(méi)配置。
查看avahi解釋：
Avahi允許程序在不需要進(jìn)行手動(dòng)網(wǎng)絡(luò)配置的情況 下，在一個(gè)本地網(wǎng)絡(luò)中發(fā)布和獲知各種服務(wù)和主機(jī)。例如，當(dāng)某用戶把他的計(jì)算機(jī)接入到某個(gè)局域網(wǎng)時(shí)，如果他的機(jī)器運(yùn)行有Avahi服務(wù)，則Avahi程式自 動(dòng)廣播，從而發(fā)現(xiàn)網(wǎng)絡(luò)中可用的打印機(jī)、共享文件和可相互聊天的其他用戶，這有點(diǎn)象他正在接收局域網(wǎng)中的各種網(wǎng)絡(luò)廣告一樣。
如此看來(lái)，可以先忽略；
 運(yùn)行：  
`  ./gridSetup.sh `
注意：先擇 網(wǎng)絡(luò)時(shí)選擇asm&private,ausize默認(rèn)為4MB了,其他安裝和以前基本相同；
查看安裝日志:
` tail -30f /data/grid/install/root_rhel7_2019-02-23_13-05-05-377719388.log`
 查看ohasd日志:

tail -30f $ORACLE_BASE/diag/crs/rhel7/crs/trace/ohasd.trc
tail -30f $ORACLE_BASE/diag/crs/rhel7/crs/trace/ohasd_orarootagent_root.trc

**10 查看資源狀態(tài)**
> [grid@rhel6 bin]$ crsctl stat res -t
> 
> Name           Target  State        Server                   State details       
> 
> Local Resources
> 
> ora.LISTENER.lsnr
>                ONLINE  ONLINE       rhel6                    STABLE
>                ONLINE  ONLINE       rhel7                    STABLE
> ora.chad
>                ONLINE  ONLINE       rhel6                    STABLE
>                ONLINE  ONLINE       rhel7                    STABLE
> ora.net1.network
>                ONLINE  ONLINE       rhel6                    STABLE
>                ONLINE  ONLINE       rhel7                    STABLE
> ora.ons
>                ONLINE  ONLINE       rhel6                    STABLE
>                ONLINE  ONLINE       rhel7                    STABLE
> 
> Cluster Resources
> 
> ora.ASMNET1LSNR_ASM.lsnr(ora.asmgroup)
>       1        ONLINE  ONLINE       rhel7                    STABLE
>       2        ONLINE  ONLINE       rhel6                    STABLE
>       3        OFFLINE OFFLINE                               STABLE
> ora.DATA.dg(ora.asmgroup)
>       1        ONLINE  ONLINE       rhel7                    STABLE
>       2        ONLINE  ONLINE       rhel6                    STABLE
>       3        OFFLINE OFFLINE                               STABLE
> ora.LISTENER_SCAN1.lsnr
>       1        ONLINE  ONLINE       rhel7                    STABLE
> ora.asm(ora.asmgroup)
>       1        ONLINE  ONLINE       rhel7                    Started,STABLE
>       2        ONLINE  ONLINE       rhel6                    Started,STABLE
>       3        OFFLINE OFFLINE                               STABLE
> ora.asmnet1.asmnetwork(ora.asmgroup)
>       1        ONLINE  ONLINE       rhel7                    STABLE
>       2        ONLINE  ONLINE       rhel6                    STABLE
>       3        OFFLINE OFFLINE                               STABLE
> ora.cvu
>       1        ONLINE  ONLINE       rhel7                    STABLE
> ora.qosmserver
>       1        ONLINE  ONLINE       rhel7                    STABLE
> ora.rhel6.vip
>       1        ONLINE  ONLINE       rhel6                    STABLE
> ora.rhel7.vip
>       1        ONLINE  ONLINE       rhel7                    STABLE
> ora.scan1.vip
>       1        ONLINE  ONLINE       rhel7                    STABLE
> 
在安裝過(guò)程中檢查內(nèi)存使用情況，其實(shí)4GB是可以 安裝完成grid的。

[grid@rhel6 bin]$ free -m
total used free shared buff/cache available
Mem: 3789 1339 386 653 2063 1579
Swap: 2047 0 2047

檢查OCR使用量：

ASMCMD> lsdg
State Type Rebal Sector Logical_Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name
MOUNTED EXTERN N 512 512 4096 4194304 5120 4784 0 4784 0 Y DATA/