메뉴 건너뛰기

Korea Oracle User Group

Install/Configuration

오라클 21c RAC 설치 with Oracle Linux 8.2 - 1 (OS 및 스토리지 설정)

 

참고용 19c RAC 설치 게시글

Oracle 19c RAC 설치 with Oracle Linux 8.2 - 1 (OS 및 스토리지 설정)

Oracle 19c RAC 설치 with Oracle Linux 8.2 - 2 (Grid Infrastructure)

Oracle 19c RAC 설치 with Oracle Linux 8.2 - 3 (사용할 Disk Group 생성)

Oracle 19c RAC 설치 with Oracle Linux 8.2 - 4 (Database Software 설치)

Oracle 19c RAC 설치 with Oracle Linux 8.2 - 5 (DBCA로 Database 생성)

 

 

0. 오라클 리눅스 설치 환경

1. 방화벽 비활성화(node 1, 2)

2. 불필요한 서비스 중지(node 1, 2)

3. 설치된 패키지 upgrade(node 1, 2)

4. /etc/hosts 파일 수정(node 1, 2)

5. Selinux 비활성화(node 1, 2)

6. chrony 설정 비활성화(node 1, 2)

7. preinstall 점검 패키지 설치(node 1, 2)

8. 추가 OS 계정 생성 및 변경

9. grid / oracle 계정의 .bash_profile 수정

10. grid / oracle /root Equivalence 설정

11. iscsi 를 통해 ASM에 사용할 디스크 설정

12. SCAN 사용을 위해 DNS 설정하기

 

0. 오라클 리눅스 설치 환경

Version : 8.2 Enterprise Linux

CPU : 4 core

Memory : 10 GB

OS : Oracle Linux 8.2

Swap : 20 GB

 

OS 설치시 Software Selection 에서 아래 내용으로 패키지 설치하였습니다.

 

 

1
2
3
4
5
6
7
8
9
10
11
12
13
Base Environment : Server with GUI
Additional software for Selected Environment :
     Debugging Tools
     File and Storage Server
     Network File System Client
     Performance Tools
     Remote Desktop Clients
     Remote Management for Linux
     Legacy UNIX Compatibility
     Development Tools
     Graphical Administration Tools
     Security Tools
     System Tools
 

 

IP 설정 : public 192.168.45.201 / 192.168.45.202

            private 1.1.1.201 / 1.1.1.202

            vip 192.168.45.211 / 192.168.45.212

            scan 192.168.45.221 / 192.168.45.222 / 192.168.45.223

 

1. 방화벽 비활성화(node 1, 2)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
[root@mprac1 ~]# systemctl stop firewalld.service
[root@mprac1 ~]# systemctl disable firewalld.service
Removed /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
[root@mprac1 ~]# systemctl status firewalld.service 
● firewalld.service - firewalld - dynamic firewall daemon
   Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled)
   Active: inactive (dead)
     Docs: man:firewalld(1)
 
11월 30 17:38:59 mprac1.localdomain systemd[1]: Starting firewalld - dynamic firewall daemon...
11월 30 17:39:01 mprac1.localdomain systemd[1]: Started firewalld - dynamic firewall daemon.
11월 30 17:39:01 mprac1.localdomain firewalld[1241]: WARNING: AllowZoneDrifting is enabled. This >
11월 30 21:55:05 mprac1.localdomain systemd[1]: Stopping firewalld - dynamic firewall daemon...
11월 30 21:55:06 mprac1.localdomain systemd[1]: firewalld.service: Succeeded.
11월 30 21:55:06 mprac1.localdomain systemd[1]: Stopped firewalld - dynamic firewall daemon.
lines 1-11/11 (END)
 

 

2. 불필요한 서비스 중지(node 1, 2)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
[root@mprac1 ~]# systemctl stop bluetooth
[root@mprac1 ~]# systemctl disable bluetooth
Removed /etc/systemd/system/dbus-org.bluez.service.
Removed /etc/systemd/system/bluetooth.target.wants/bluetooth.service.
[root@mprac1 ~]# systemctl stop avahi-daemon.socket avahi-daemon.service
[root@mprac1 ~]# systemctl disable avahi-daemon.socket avahi-daemon.service
Removed /etc/systemd/system/multi-user.target.wants/avahi-daemon.service.
Removed /etc/systemd/system/sockets.target.wants/avahi-daemon.socket.
Removed /etc/systemd/system/dbus-org.freedesktop.Avahi.service.
[root@mprac1 ~]# systemctl mask avahi-daemon.socket avahi-daemon.service
Created symlink /etc/systemd/system/avahi-daemon.socket → /dev/null.
Created symlink /etc/systemd/system/avahi-daemon.service → /dev/null.
[root@mprac1 ~]# systemctl stop libvirtd.service
Warning: Stopping libvirtd.service, but it can still be activated by:
  libvirtd-admin.socket
  libvirtd.socket
  libvirtd-ro.socket
[root@mprac1 ~]# systemctl disable libvirtd.service
Removed /etc/systemd/system/multi-user.target.wants/libvirtd.service.
Removed /etc/systemd/system/sockets.target.wants/virtlogd.socket.
Removed /etc/systemd/system/sockets.target.wants/virtlockd.socket.
 

 

3. 설치된 패키지 upgrade(node 1, 2)

1
2
3
4
5
[root@mprac1 ~]# dnf upgrade
마지막 메타자료 만료확인 1:07:11 이전인: 2021년 11월 30일 (화) 오후 09시 09분 14초.
종속성이 해결되었습니다.
처리가 필요하지 않습니다.
완료되었습니다!
 

 

4. /etc/hosts 파일 수정(node 1, 2)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
[root@mprac1 ~]# vi /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
 
# Public
192.168.45.201 mprac1 mprac1.localdomain
192.168.45.202 mprac2 mprac2.localdomain
 
# Private
1.1.1.201 mprac1-priv mprac1-priv.localdomain
1.1.1.202 mprac2-priv mprac2-priv.localdomain
 
# VIP
192.168.45.211 mprac1-vip mprac1-vip.localdomain
192.168.45.212 mprac2-vip mprac2-vip.localdomain
 
# iscsi Storage Server
192.168.45.105 kwanst kwanst.localdomain
1.1.1.105 kwanst-priv kwanst-priv.localdomain
 

 

5. Selinux 비활성화(node 1, 2)

1
2
3
4
5
6
7
8
9
10
11
12
13
[root@mprac1 ~]# vi /etc/selinux/config
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
#SELINUX=enforcing
SELINUX=disabled
# SELINUXTYPE= can take one of these three values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected.
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted
 

 

변경 후 시스템 reboot 후 변경 사항 확인

 

1
2
3
4
5
[root@mprac1 ~]# getenforce
Disabled
[root@mprac1 ~]# sestatus -v
SELinux status:                 disabled
[root@mprac1 ~]# 
 

 

6. chrony 설정 비활성화(node 1, 2)

1
2
3
[root@mprac1 ~]# systemctl stop chronyd
[root@mprac1 ~]# systemctl disable chronyd
[root@mprac1 ~]# mv /etc/chrony.conf /etc/chrony.conf.bak
 

 

7. preinstall 점검 패키지 설치(node 1, 2)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
[root@mprac1 ~]# dnf list oracle-database-preinstall*
마지막 메타자료 만료확인 1:29:57 이전인: 2021년 11월 30일 (화) 오후 09시 09분 14초.
사용 가능한 꾸러미
oracle-database-preinstall-19c.src                      1.0-2.el8                   ol8_appstream
oracle-database-preinstall-19c.x86_64                   1.0-2.el8                   ol8_appstream
oracle-database-preinstall-21c.src                      1.0-1.el8                   ol8_appstream
oracle-database-preinstall-21c.x86_64                   1.0-1.el8                   ol8_appstream
[root@mprac1 ~]# dnf install oracle-database-preinstall-21c
마지막 메타자료 만료확인 1:31:06 이전인: 2021년 11월 30일 (화) 오후 09시 09분 14초.
종속성이 해결되었습니다.
=================================================================================================
 꾸러미                            구조      버전                     레포지터리            크기
=================================================================================================
설치 중:
 oracle-database-preinstall-21c    x86_64    1.0-1.el8                ol8_appstream         30 k
종속 꾸러미 설치 중:
 compat-openssl10                  x86_64    1:1.0.2o-3.el8           ol8_appstream        1.1 M
 ksh                               x86_64    20120801-254.0.1.el8     ol8_appstream        927 k
 libnsl                            x86_64    2.28-164.0.1.el8         ol8_baseos_latest    104 k
 
연결 요약
=================================================================================================
설치  4 꾸러미
 
총계 내려받기 크기: 2.2 M
설치된 크기 : 6.6 M
진행 할 까요? [y/N]: y
꾸러미 내려받기중:
(1/4): libnsl-2.28-164.0.1.el8.x86_64.rpm                        177 kB/| 104 kB     00:00    
(2/4): oracle-database-preinstall-21c-1.0-1.el8.x86_64.rpm       4.8 MB/|  30 kB     00:00    
(3/4): compat-openssl10-1.0.2o-3.el8.x86_64.rpm                  1.8 MB/| 1.1 MB     00:00    
(4/4): ksh-20120801-254.0.1.el8.x86_64.rpm                       1.4 MB/| 927 kB     00:00    
-------------------------------------------------------------------------------------------------
합계                                                             3.3 MB/| 2.2 MB     00:00     
연결 확인 실행 중
연결 확인에 성공했습니다.
연결 시험 실행 중
연결 시험에 성공했습니다.
연결 실행 중
  준비 중           :                                                                        1/1 
  설치 중           : ksh-20120801-254.0.1.el8.x86_64                                        1/4 
  스크립트릿 실행 중: ksh-20120801-254.0.1.el8.x86_64                                        1/4 
  설치 중           : compat-openssl10-1:1.0.2o-3.el8.x86_64                                 2/4 
  스크립트릿 실행 중: compat-openssl10-1:1.0.2o-3.el8.x86_64                                 2/4 
  설치 중           : libnsl-2.28-164.0.1.el8.x86_64                                         3/4 
  설치 중           : oracle-database-preinstall-21c-1.0-1.el8.x86_64                        4/4 
  스크립트릿 실행 중: oracle-database-preinstall-21c-1.0-1.el8.x86_64                        4/4 
  확인 중           : libnsl-2.28-164.0.1.el8.x86_64                                         1/4 
  확인 중           : compat-openssl10-1:1.0.2o-3.el8.x86_64                                 2/4 
  확인 중           : ksh-20120801-254.0.1.el8.x86_64                                        3/4 
  확인 중           : oracle-database-preinstall-21c-1.0-1.el8.x86_64                        4/4 
 
설치되었습니다:
  compat-openssl10-1:1.0.2o-3.el8.x86_64     ksh-20120801-254.0.1.el8.x86_64                    
  libnsl-2.28-164.0.1.el8.x86_64             oracle-database-preinstall-21c-1.0-1.el8.x86_64    
 
완료되었습니다!
 

 

패키지가 설치되면 오라클 설치에 필요한 설정에 대해서 변경 작업을 수행하고 해당 내용의 로그를 아래 파일에 기록하게 된다.

/var/log/oracle-database-preinstall-21c/backup/Nov-30-2021-22-40-41/orakernel.log

로그 내용을 확인하고 체크해야한다.

 

패키지가 설치 수행되면 설치에 필요한 계정과 그룹이 생성되게 된다.

 

8. 추가 OS 계정 생성 및 변경

아래 내용으로 modify_oracle_user.sh shell script 생성

 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
[root@mprac1 ~]# vi modify_oracle_user.sh
 
#!/usr/bin/ksh
# Program : modify_oracle_users.sh
# Author : Sean Francis
# Date : July 2019
# Purpose : To complete the setup of the oracle and grid users for a new
# server install.
#
# Notes : This script assumes the pre-install database package has been
# installed.
################################################################################
# STANDARD ORACLE USER AND GROUP IDs
#
# USERS:
ORACLE_UID=1000
GRID_UID=1001
# GROUPS:
OINSTALL_GID=1001
DBA_GID=1002
OPER_GID=1003
BACKUPDBA_GID=1004
DGDBA_GID=1005
KMDBA_GID=1006
RACDBA_GID=1007
ASMDBA_GID=1008
ASMADMIN_GID=1009
ASMOPER_GID=1010
################################################################################
# add group
groupmod -g ${OINSTALL_GID} oinstall
groupmod -g ${DBA_GID} dba
groupmod -g ${OPER_GID} oper
groupmod -g ${BACKUPDBA_GID} backupdba
groupmod -g ${DGDBA_GID} dgdba
groupmod -g ${KMDBA_GID} kmdba
groupmod -g ${RACDBA_GID} racdba
 
# add missing ASM groups
groupadd -g ${ASMDBA_GID} asmdba
groupadd -g ${ASMADMIN_GID} asmadmin
groupadd -g ${ASMOPER_GID} asmoper
 
# modify oracle
usermod -u ${ORACLE_UID} -g oinstall -G dba,backupdba,dgdba,kmdba,racdba,asmdba,oper oracle
 
# add grid
useradd -u ${GRID_UID} -g oinstall -G dba,asmdba,asmadmin,asmoper,racdba grid
 
# display the new user configurations
echo ""
echo "oracle user id: " `id oracle`
echo ""
echo "grid user id: " `id grid`
# eof modify_oracle_users.sh
 

 

생성한 shell 스크립트를 수행한다.

 

1
2
3
4
5
6
[root@mprac1 ~]# chmod +x modify_oracle_user.sh 
[root@mprac1 ~]# sh modify_oracle_user.sh 
 
oracle user id:  uid=1000(oracle) gid=1001(oinstall) groups=1001(oinstall),1002(dba),1003(oper),1004(backupdba),1005(dgdba),1006(kmdba),1007(racdba),1008(asmdba)
 
grid user id:  uid=1001(grid) gid=1001(oinstall) groups=1001(oinstall),1002(dba),1007(racdba),1008(asmdba),1009(asmadmin),1010(asmoper)
 

 

grid 유저와 oracle 유저 패스워드 변경한다.

 

1
2
3
4
5
6
7
8
9
10
11
12
[root@mprac1 ~]# passwd grid
grid 사용자의 비밀 번호 변경 중
새  암호:
잘못된 암호: 암호는 8 개의 문자 보다 짧습니다
새  암호 재입력:
passwd: 모든 인증 토큰이 성공적으로 업데이트 되었습니다.
[root@mprac1 ~]# passwd oracle
oracle 사용자의 비밀 번호 변경 중
새  암호:
잘못된 암호: 암호는 8 개의 문자 보다 짧습니다
새  암호 재입력:
passwd: 모든 인증 토큰이 성공적으로 업데이트 되었습니다.
 

 

9. grid / oracle 계정의 .bash_profile 수정

grid 유저의 .bash_profile 수정

아래 내용을 추가해 준다.

 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
stty erase ^H
umask 022
ORACLE_SID=+ASM1
ORACLE_BASE=/u01/app/grid
ORACLE_HOME=/u01/app/21c/grid
LD_LIBRARY_PATH=$ORACLE_HOME/lib
TMP=/tmp
TMPDIR=/tmp
PATH=$PATH:$HOME/.local/bin:$HOME/bin:$ORACLE_HOME/bin
export PATH ORACLE_SID ORACLE_BASE ORACLE_HOME LD_LIBRARY_PATH TMP TMPDIR
export CV_ASSUME_DISTID=OEL8.1 #for Linux 8 ver
export ORA_DISABLED_CVU_CHECKS="TASKNTP,TASKCTSSINTEGRITY" #for ntp error solution
 
#export SQLPATH=
export PS1=`hostname`'@$LOGNAME:$ORACLE_SID:$PWD> '
 
alias sqlss='sqlplus / as sysdba'
alias ob="cd ${ORACLE_BASE}"
alias oh="cd ${ORACLE_HOME}"
 

 

2번 노드의 경우 +ASM1을 +ASM2로 변경해야 한다.

 

oracle 유저의 .bash_profile 수정

아래 내용을 추가해 준다.

 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
stty erase ^H
umask 022
ORACLE_SID=MPRAC1
ORACLE_BASE=/u01/app/oracle
ORACLE_HOME=$ORACLE_BASE/product/21c/db_1
LD_LIBRARY_PATH=$ORACLE_HOME/lib
TMP=/tmp
TMPDIR=/tmp
PATH=$PATH:$HOME/.local/bin:$HOME/bin:$ORACLE_HOME/bin
export PATH ORACLE_SID ORACLE_BASE ORACLE_HOME LD_LIBRARY_PATH TMP TMPDIR
export CV_ASSUME_DISTID=OEL8.1 #for Linux 8 ver
 
#export SQLPATH=
export PS1=`hostname`'@$LOGNAME:$ORACLE_SID:$PWD> '
 
alias sqlss='sqlplus / as sysdba'
alias ob="cd ${ORACLE_BASE}"
alias oh="cd ${ORACLE_HOME}"
 

 

2번 노드의 경우 MPRAC1을 MPRAC2로 변경해야 한다.

 

root 유저의 .bash_profile 수정

아래 내용 추가

 

1
2
3
4
5
6
7
8
9
10
11
ORACLE_BASE=/u01/app/grid
ORACLE_HOME=/u01/app/21c/grid
 
PATH=$PATH:$HOME/bin:$ORACLE_HOME/bin
 
export PATH ORACLE_BASE ORACLE_HOME
export PS1=`hostname`'@$LOGNAME:$PWD> '
 
alias sqlss='sqlplus / as sysdba'
alias gb="cd ${ORACLE_BASE}"
alias gh="cd ${ORACLE_HOME}"
 

 

10. grid / oracle /root Equivalence 설정

grid 유저 세팅

 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
[root@mprac1 ~]# su - grid
mprac1.localdomain@grid:+ASM1:/home/grid> rm -rf ~/.ssh
mprac1.localdomain@grid:+ASM1:/home/grid> mkdir ~/.ssh
mprac1.localdomain@grid:+ASM1:/home/grid> chmod 700 ~/.ssh
mprac1.localdomain@grid:+ASM1:/home/grid> cd ~/.ssh
mprac1.localdomain@grid:+ASM1:/home/grid/.ssh> ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/grid/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/grid/.ssh/id_rsa.
Your public key has been saved in /home/grid/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:0SwpHgbWEX2Ls4hUzDAjM7nLBuI3MPq1oHr2heBEDRs grid@mprac1.localdomain
The key's randomart image is:
+---[RSA 3072]----+
|  E.*=o+         |
|  .X += .+.      |
|  o...+ +oo.     |
|o+. .o ooo.      |
|+o=o ...So       |
|.+=+o.. .        |
| +ooo..          |
|. + ..           |
|oo ..            |
+----[SHA256]-----+
mprac1.localdomain@grid:+ASM1:/home/grid/.ssh> ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/home/grid/.ssh/id_dsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/grid/.ssh/id_dsa.
Your public key has been saved in /home/grid/.ssh/id_dsa.pub.
The key fingerprint is:
SHA256:Cip74o6baI/eJgfQanX94JL7koVG+zoyoL+zRZV5fgA grid@mprac1.localdomain
The key's randomart image is:
+---[DSA 1024]----+
|      E          |
|       +         |
| .    = o        |
|. .. + = .       |
|... = = S .      |
|.+ o B + o       |
|+ + o B          |
|=*=* = .         |
|@XO*o.=.         |
+----[SHA256]-----+
mprac1.localdomain@grid:+ASM1:/home/grid/.ssh>
 

 

2번 노드에서도 동일한 작업 수행한다.

 

1번 노드에서 아래 작업을 먼저 수행해 준다.

 

1
2
3
4
5
6
7
8
9
mprac1.localdomain@grid:+ASM1:/home/grid/.ssh> cat *.pub >> authorized_keys
mprac1.localdomain@grid:+ASM1:/home/grid/.ssh> scp authorized_keys mprac2:~/.ssh/
The authenticity of host 'mprac2 (192.168.45.202)' can't be established.
ECDSA key fingerprint is SHA256:da+T4x2FNHO9O+wgULy9DFf/ESx85SLTUNq/q2dw/qo.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added 'mprac2,192.168.45.202' (ECDSA) to the list of known hosts.
grid@mprac2's password: 
authorized_keys                                                1001190   813.6KB/s   00:00    
mprac1.localdomain@grid:+ASM1:/home/grid/.ssh>
 

 

2번 노드에서 아래 작업을 수행해 준다.

 

1
2
3
4
5
6
7
8
9
mprac2.localdomain@grid:+ASM2:/home/grid/.ssh> cat *.pub >> authorized_keys 
mprac2.localdomain@grid:+ASM2:/home/grid/.ssh> scp authorized_keys mprac1:~/.ssh
The authenticity of host 'mprac1 (192.168.45.201)' can't be established.
ECDSA key fingerprint is SHA256:qkXvzJMHYVSSm8xEU6h9CE8GgdOeFDwjY0qc0BeDYG4.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added 'mprac1,192.168.45.201' (ECDSA) to the list of known hosts.
grid@mprac1's password: 
authorized_keys                                                1002380     2.4MB/s   00:00    
mprac2.localdomain@grid:+ASM2:/home/grid/.ssh>
 

 

oracle 유저 세팅

grid 유저 세팅과 같은 동일 작업을 oracle 계정에서 수행한다.

 

root 유저 세팅

grid 유저 세팅과 같은 동일 작업을 root 계정에서 수행한다

 

11. iscsi 를 통해 ASM에 사용할 디스크 설정

RAC에서 사용할 disk 추가

아래 링크의 내용을 참조하여 노드 1,2 에 디스크 추가 작업 수행

 

iscsi 서버/클라이언트 설정하기(with targetcli)

 

추가된 디스크에 대해서 RAC에서 사용할 수 있게 추가 설정

 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
mprac1.localdomain@root:/root> lsblk
NAME               MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda                  8:0    0   200G  0 disk 
├─sda1               8:1    0   600M  0 part /boot/efi
├─sda2               8:2    0     1G  0 part /boot
└─sda3               8:3    0 198.4G  0 part 
  ├─ol_mprac1-root 252:0    0    50G  0 lvm  /
  ├─ol_mprac1-swap 252:1    0    20G  0 lvm  [SWAP]
  └─ol_mprac1-home 252:2    0 128.4G  0 lvm  /home
sdb                  8:16   0     5G  0 disk 
sdc                  8:32   0    44G  0 disk 
sdd                  8:48   0   340G  0 disk 
sde                  8:64   0     5G  0 disk 
sdf                  8:80   0     5G  0 disk 
sr0                 11:0    1  1024M  0 rom  
mprac1.localdomain@root:/root> fdisk /dev/sdb
 
Welcome to fdisk (util-linux 2.32.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
 
Device does not contain a recognized partition table.
Created a new DOS disklabel with disk identifier 0x8b278697.
 
Command (m for help): n
Partition type
   p   primary (0 primary, 0 extended, 4 free)
   e   extended (container for logical partitions)
Select (default p): 
 
Using default response p.
Partition number (1-4, default 1): 
First sector (65528-10485759, default 65528): 
Last sector, +sectors or +size{K,M,G,T,P} (65528-10485759, default 10485759): 
 
Created a new partition 1 of type 'Linux' and of size 5 GiB.
Partition #1 contains a oracleasm signature.
 
Do you want to remove the signature? [Y]es/[N]o: Y
 
The signature will be removed by a write command.
 
Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.
 
 
mprac1.localdomain@root:/root> fdisk /dev/sdc
 
Welcome to fdisk (util-linux 2.32.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
 
Device does not contain a recognized partition table.
Created a new DOS disklabel with disk identifier 0x90123789.
 
Command (m for help): n
Partition type
   p   primary (0 primary, 0 extended, 4 free)
   e   extended (container for logical partitions)
Select (default p): 
 
Using default response p.
Partition number (1-4, default 1): 
First sector (65528-92274687, default 65528): 
Last sector, +sectors or +size{K,M,G,T,P} (65528-92274687, default 92274687): 
 
Created a new partition 1 of type 'Linux' and of size 44 GiB.
 
Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.
 
mprac1.localdomain@root:/root> fdisk /dev/sdd
 
Welcome to fdisk (util-linux 2.32.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
 
Device does not contain a recognized partition table.
Created a new DOS disklabel with disk identifier 0x8ba508b1.
 
Command (m for help): n
Partition type
   p   primary (0 primary, 0 extended, 4 free)
   e   extended (container for logical partitions)
Select (default p): 
 
Using default response p.
Partition number (1-4, default 1): 
First sector (65528-713031679, default 65528): 
Last sector, +sectors or +size{K,M,G,T,P} (65528-713031679, default 713031679): 
 
Created a new partition 1 of type 'Linux' and of size 340 GiB.
Partition #1 contains a oracleasm signature.
 
Do you want to remove the signature? [Y]es/[N]o: Y
 
The signature will be removed by a write command.
 
Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.
 
mprac1.localdomain@root:/root> fdisk /dev/sde
 
Welcome to fdisk (util-linux 2.32.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
 
Device does not contain a recognized partition table.
Created a new DOS disklabel with disk identifier 0xc57a285d.
 
Command (m for help): n
Partition type
   p   primary (0 primary, 0 extended, 4 free)
   e   extended (container for logical partitions)
Select (default p): 
 
Using default response p.
Partition number (1-4, default 1): 
First sector (65528-10485759, default 65528): 
Last sector, +sectors or +size{K,M,G,T,P} (65528-10485759, default 10485759): 
 
Created a new partition 1 of type 'Linux' and of size 5 GiB.
 
Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.
 
mprac1.localdomain@root:/root> fdisk /dev/sdf
 
Welcome to fdisk (util-linux 2.32.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
 
Device does not contain a recognized partition table.
Created a new DOS disklabel with disk identifier 0x26ff5211.
 
Command (m for help): n
Partition type
   p   primary (0 primary, 0 extended, 4 free)
   e   extended (container for logical partitions)
Select (default p): 
 
Using default response p.
Partition number (1-4, default 1): 
First sector (65528-10485759, default 65528): 
Last sector, +sectors or +size{K,M,G,T,P} (65528-10485759, default 10485759): 
 
Created a new partition 1 of type 'Linux' and of size 5 GiB.
Partition #1 contains a oracleasm signature.
 
Do you want to remove the signature? [Y]es/[N]o: Y
 
The signature will be removed by a write command.
 
Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.
 
mprac1.localdomain@root:/root> partprobe -s
/dev/sda: gpt partitions 1 2 3
/dev/sdb: msdos partitions 1
/dev/sdc: msdos partitions 1
/dev/sdd: msdos partitions 1
/dev/sde: msdos partitions 1
/dev/sdf: msdos partitions 1
mprac1.localdomain@root:/root> lsblk
NAME               MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda                  8:0    0   200G  0 disk 
├─sda1               8:1    0   600M  0 part /boot/efi
├─sda2               8:2    0     1G  0 part /boot
└─sda3               8:3    0 198.4G  0 part 
  ├─ol_mprac1-root 252:0    0    50G  0 lvm  /
  ├─ol_mprac1-swap 252:1    0    20G  0 lvm  [SWAP]
  └─ol_mprac1-home 252:2    0 128.4G  0 lvm  /home
sdb                  8:16   0     5G  0 disk 
└─sdb1               8:17   0     5G  0 part 
sdc                  8:32   0    44G  0 disk 
└─sdc1               8:33   0    44G  0 part 
sdd                  8:48   0   340G  0 disk 
└─sdd1               8:49   0   340G  0 part 
sde                  8:64   0     5G  0 disk 
└─sde1               8:65   0     5G  0 part 
sdf                  8:80   0     5G  0 disk 
└─sdf1               8:81   0     5G  0 part 
sr0                 11:0    1  1024M  0 rom  
 

 

2번 노드에서 1번 작업을 아래와 같이 인식 시켜준다.

 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
mprac2.localdomain@root:/root> lsblk
NAME               MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda                  8:0    0   200G  0 disk 
├─sda1               8:1    0   600M  0 part /boot/efi
├─sda2               8:2    0     1G  0 part /boot
└─sda3               8:3    0 198.4G  0 part 
  ├─ol_mprac2-root 252:0    0    50G  0 lvm  /
  ├─ol_mprac2-swap 252:1    0    20G  0 lvm  [SWAP]
  └─ol_mprac2-home 252:2    0 128.4G  0 lvm  /home
sdb                  8:16   0     5G  0 disk 
sdc                  8:32   0    44G  0 disk 
sdd                  8:48   0   340G  0 disk 
sde                  8:64   0     5G  0 disk 
sdf                  8:80   0     5G  0 disk 
sr0                 11:0    1  1024M  0 rom  
mprac2.localdomain@root:/root> partprobe -s
/dev/sda: gpt partitions 1 2 3
/dev/sdb: msdos partitions 1
/dev/sdc: msdos partitions 1
/dev/sdd: msdos partitions 1
/dev/sde: msdos partitions 1
/dev/sdf: msdos partitions 1
mprac2.localdomain@root:/root> lsblk
NAME               MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda                  8:0    0   200G  0 disk 
├─sda1               8:1    0   600M  0 part /boot/efi
├─sda2               8:2    0     1G  0 part /boot
└─sda3               8:3    0 198.4G  0 part 
  ├─ol_mprac2-root 252:0    0    50G  0 lvm  /
  ├─ol_mprac2-swap 252:1    0    20G  0 lvm  [SWAP]
  └─ol_mprac2-home 252:2    0 128.4G  0 lvm  /home
sdb                  8:16   0     5G  0 disk 
└─sdb1               8:17   0     5G  0 part 
sdc                  8:32   0    44G  0 disk 
└─sdc1               8:33   0    44G  0 part 
sdd                  8:48   0   340G  0 disk 
└─sdd1               8:49   0   340G  0 part 
sde                  8:64   0     5G  0 disk 
└─sde1               8:65   0     5G  0 part 
sdf                  8:80   0     5G  0 disk 
└─sdf1               8:81   0     5G  0 part 
sr0                 11:0    1  1024M  0 rom  
 

 

서버 리부팅시 disk 경로가 변경되는 것을 방지하기 위해 udev 설정

먼저 scsi_id 명령어를 통해 SCSI identifier를 확인한다.

 

1
2
3
4
5
6
7
8
9
10
11
mprac1.localdomain@root:/root> alias scsi_id='/lib/udev/scsi_id'
mprac1.localdomain@root:/root> scsi_id ---/dev/sdb
3600140520c5741d76894d129d6e7ec64
mprac1.localdomain@root:/root> scsi_id ---/dev/sdc
360014055f7ab396e8a94602a13f0da8a
mprac1.localdomain@root:/root> scsi_id ---/dev/sdd
360014051c2d63a348d14a09bc3ede1de
mprac1.localdomain@root:/root> scsi_id ---/dev/sde
36001405c5cf72f22bb54ebca403a2134
mprac1.localdomain@root:/root> scsi_id ---/dev/sdf
360014059642c6c68b1047d8936c46963
 

 

udev 설정을 위해 아래 내용으로 신규 rule를 추가해 준다.

5G 사이즈의 disk는 OCR / VOTING용으로 사용하고 다른 사이즈의 disk는 각 data 영역과 fra 영역으로 사용하도록 설정한다.

 

1
2
3
4
5
6
7
mprac2.localdomain@root:/root> vi /lib/udev/rules.d/99-iscsidevices.rules
 
KERNEL=="sd?1", SUBSYSTEM=="block", PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$parent", RESULT=="3600140520c5741d76894d129d6e7ec64", SYMLINK+="rac/crs1", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd?1", SUBSYSTEM=="block", PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$parent", RESULT=="36001405c5cf72f22bb54ebca403a2134", SYMLINK+="rac/crs2", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd?1", SUBSYSTEM=="block", PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$parent", RESULT=="360014059642c6c68b1047d8936c46963", SYMLINK+="rac/crs3", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd?1", SUBSYSTEM=="block", PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$parent", RESULT=="360014051c2d63a348d14a09bc3ede1de", SYMLINK+="rac/data", OWNER="grid", GROUP="asmadmin", MODE="0660"
KERNEL=="sd?1", SUBSYSTEM=="block", PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$parent", RESULT=="360014055f7ab396e8a94602a13f0da8a", SYMLINK+="rac/fra", OWNER="grid", GROUP="asmadmin", MODE="0660"
 

 

아래 명령어를 통해 테스트 후 device 명이 설정되었는지 확인한다.

 

1
2
3
4
5
6
7
8
mprac1.localdomain@root:/root> udevadm test /block/sdb/sdb1
..
..
mprac1.localdomain@root:/root> ls -al /dev/rac
합계 0
drwxr-xr-x  2 root root   60 12월  1 10:59 .
drwxr-xr-20 root root 3540 12월  1 10:59 ..
lrwxrwxrwx  1 root root    7 12월  1 10:59 crs -> ../sdb1
 

 

작업 후 시스템 reboot 후 아래와 같이 다시 확인해 본다.

 

1
2
3
4
5
6
7
8
9
mprac1.localdomain@root:/root> ls -al /dev/rac
합계 0
drwxr-xr-x  2 root root  140 12월  2 11:26 .
drwxr-xr-20 root root 3660 12월  2 11:26 ..
lrwxrwxrwx  1 root root    7 12월  2 11:26 crs1 -> ../sdb1
lrwxrwxrwx  1 root root    7 12월  2 11:26 crs2 -> ../sde1
lrwxrwxrwx  1 root root    7 12월  2 11:26 crs3 -> ../sdf1
lrwxrwxrwx  1 root root    7 12월  2 11:26 data -> ../sdd1
lrwxrwxrwx  1 root root    7 12월  2 11:26 fra -> ../sdc1
 

 

12. SCAN 사용을 위해 DNS 설정하기

아래 링크된 게시글을 참조하기 바랍니다.

 

RAC SCAN 사용을 위한 DNS 서버 설정하기

 

 

 

 

 

번호 제목 글쓴이 날짜 조회 수
40 Oracle Database 19c Patch Update(단일 인스턴스 오라클 DB 19.3 에서 19.9로 RU Update) 명품관 2020.11.24 34923
39 New version of our Hands-On Lab available for download "Upgrade/Migrate/Consolidate to Oracle 12c" HOL 명품관 2015.12.22 8992
38 How to Enable/Disable Oracle Binary Option [1] Talros 2016.03.09 8620
37 19c RAC OJVM 패치작업 우뽕 2020.04.22 5209
36 Oracle RAC RU rolling patching from 19.3 to 19.9.0.0.201020(Patch 31750108) [1] file 명품관 2020.12.08 5153
35 Oracle 19c RAC 설치 with Oracle Linux 8.2 - 2 (Grid Infrastructure) file 명품관 2020.11.19 4935
34 Oracle 19c RAC 설치 with Oracle Linux 8.2 - 1 (OS 및 스토리지 설정) 명품관 2020.11.14 4743
33 ASM 에 spfile 등록 및 삭제 하기 우뽕 2020.07.27 4433
» 오라클 21c RAC 설치 with Oracle Linux 8.2 - 1 (OS 및 스토리지 설정) 명품관 2021.12.01 2988
31 Oracle 19c RAC 설치 with Oracle Linux 8.2 - 5 (DBCA로 Database 생성) file 명품관 2020.11.19 2805
30 19c RAC 2Node 설치 -1 [ 환경 ,OS 세팅 ] file 우뽕 2020.04.14 2647
29 19c RAC - Manual Patch 적용방법 우뽕 2020.05.05 2334
28 Oracle EM(Enterprise Manager) 13.4 설치 및 구성 - 2(EM 설치) file 명품관 2020.11.27 2241
27 Oracle 19c RAC 설치 with Oracle Linux 8.2 - 3 (사용할 Disk Group 생성) file 명품관 2020.11.19 1478
26 DBCA/DBUA 사용시 마지막 단계 Finish 버튼 클릭 후 행 현상 발생 시 체크 사항 명품관 2015.12.17 1460
25 19c RAC 2Node 설치-8 [ Patch ] 우뽕 2020.04.22 1396
24 19c RAC 2Node 삭제 하기 file 우뽕 2020.04.22 1336
23 메뉴얼하게 DB 추가 작업 - GRID 리소스에 DB 추가 작업 우뽕 2020.07.27 1252
22 Oracle EM(Enterprise Manager) 13.4 설치 및 구성 - 1(Repository DB 설치) file 명품관 2020.11.23 1146
21 Automatic Storage Management (ASM) 사용시 고려해야 할 사항 [2] 우뽕 2020.02.06 1041
위로