- Arun Kumar
Oracle 11gR2 RAC Installation on Oracle Linux 6.5
In this article we will be installing 2 node 11gR2 RAC on Oracle Linux 6.5. We will be looking at below steps:
Configure VM for Node 1
Open virtual box and create a new VM for node 1. We will be installing OEL 6.5 on both RAC nodes.
Create New VM
Open virtual box and click on New. Create first virtual machine with oraracn1 as the name.
Min RAM allocation should be 4 GB per RAC node.
Create a virtual disk now
Select VDI and click next
Select Dynamically allocated option and click next
Allocate Min 80 GB HDD to your virtual machine. You can allocate even more if you have good hard disk space. Click on Create
Now your virtual machine should be created. This is how your virtual box will look like
Right click on virtual machine and select settings. Under System settings, make sure to set below bootorder
Under storage > Optical Drive > choose OEL 6.5 iso file from your system.
Under Network, make sure attached to is set to Bridged Adapter and Enable the Network Adapter
In RAC, we need to have two network adapters. So, enable the second adapter same like the first one.
Click on OK to complete the RAC node 1 setup.
Install OEL 6.5 on Node 1
Now that our virtual machine is ready, go ahead, start the virtual machine and install OEL 6.5
Skip media test
Click next to continue
Go with default language > click next
Go with default keyboard type and click on next
Select Basic Storage Devices > Click Next
Select Yes, Discard and data
Give a hostname to your machine oraracn1.oraclgenesis.com
Click on configure network and assign a static IP
Assign public IP to your machine. In my case, I have assigned 192.168.0.101 as the machine IP.
Select second network card and click on edit
Assign private IP to your node 1 on a different subnet mask. Leave Gateway blank
Click next and select your timezone
Give a root password
Select create custom layout and click next
Select Free disk space and click on create
Choose standard partition and click Create
Create boot partition with 512 MB size
Once again choose free disk space and click on create
Select file system type as Swap and give 15 GB size (Approx 15000 MB)
Again select free space and click on create
Give mount point as root (/) and chose option Fill to maximum allowable size
Click next and chose Forma option
Select Write changes to disk
No changes on below screen, just click next
Choose customize now and just click next
From the list, select packages as below:
Base System:
Base
Client management tools
Compatibility libraries
Hardware monitoring utilities
Large Systems Performance
Network file system client
Performance Tools
Perl Support
Servers:
Server Platform
System administration tools
Desktops
Desktop
Desktop Platform
Fonts
General Purpose Desktop
Graphical Administration Tools
Input Methods
X Window System
Development:
Additional Development
Development Tools
Applications:
Internet Browser
Click next and OEL installation will begin
Once done, click on Reboot
After reboot, there are few post installation steps. Click on Forward
Accept the license and click on Forward
Choose No and click on Forward
Select No thanks and proceed
Click forward
Do not create any user at this stage, click forward
Choose Yes
Check the Date and Time settings and click on forward
Nothing to choose under Kdump, click forward
OEL 6.5 installation is completed.
Configure VM for Node 2
You need to create Node 2 like how you have created Node 1 in virtual box. This will be a separate new virtual machine. I have used below configuration for node 2:
Install OEL 6.5 on Node 2
Now that our virtual machine is ready, go ahead and start the virtual machine.
Start virtual machine and install OEL 6.5
Skip media test
Click next to continue
Go with default language > click next
Go with default keyboard type and click on next
Select Basic Storage Devices > Click Next
Select Yes, Discard and data
Give a hostname to your machine oraracn2.oraclgenesis.com
Click on configure network and assign a static IP
Assign public IP to your machine. In my case, I have assigned 192.168.0.102 as the machine IP.
Select second network card and click on edit
Assign private IP to your node 2 on a different subnet mask. Leave Gateway blank
Click next and select your timezone
Give a root password
Select create custom layout and click next
Select Free disk space and click on create
Choose standard partition and click Create
Create boot partition with 512 MB size
Once again choose free disk space and click on create
Select file system type as Swap and give 15 GB size (Approx 15000 MB)
Again select free space and click on create
Give mount point as root (/) and chose option Fill to maximum allowable size
Click next and chose Format option
Select Write changes to disk
No changes on below screen, just click next
Choose customize now and just click next
From the list, select packages as below:
Base System:
Servers:Desktops:
Desktops:
Applications:
Development:
Click next and OEL installation will begin
Once done, click on Reboot
After reboot, there are few post installation steps. Click on Forward
Accept the license and click on Forward
Choose No and click on Forward
Select No thanks and proceed
Click forward
Do not create any user at this stage, click forward
Choose Yes
Check the Date and Time settings and click on forward
Nothing to choose under Kdump, click forward
OEL 6.5 installation on Node 2 is completed.
OS Prerequisites – All Nodes
Connect to both RAC nodes via putty and execute all the below commands on both nodes.
Check if your machine is able to speak to internet. You can ping yahoo.com. Then install below package to perform all oracle installation pre-requisites
yum -y install oracle-rdbms*
Install ASM packages
yum -y install oracleasm* yum -y install kmod-oracleasm
Change password for oracle user
passwd oracle
Disable firewall
/etc/rc.d/init.d/iptables status /etc/rc.d/init.d/iptables stop service iptables stop chkconfig iptables off echo 0 > /selinux/enforce
Create directories to install grid and oracle software
mkdir -p /u01/app/11.2.0/grid mkdir -p /u01/app/oracle/product/11.2.0/db_1 chown -R oracle:oinstall /u01 chmod -R 775 /u01/ mkdir -p /u02 chown -R oracle:oinstall /u02
Download Oracle 11.2.0.4 gird and database binaries. Copy it to /u02 location (Only on Node 1)
Setup Public & Private Network
We have below public and private network configuration for our RAC setup:
Also, below are the Virtual IP and Scan IP details:
Depending upon your environment, change IP addresses and put below under /etc/hosts file on both oraracn1 and oraracn2 nodes
On both nodes: ============== 127.0.0.1 localhost.localdomain localhost # Public Network - (eth0) 192.168.0.101 oraracn1.oraclegenesis.com oraracn1 192.168.0.102 oraracn2.oraclegenesis.com oraracn2 # Private Interconnect - (eth1) 10.10.10.101 oraracn1-priv.oraclegenesis.com oraracn1-priv 10.10.10.102 oraracn2-priv.oraclegenesis.com oraracn2-priv # Public Virtual IP (VIP) addresses - (eth0:1) 192.168.0.105 oraracn1-vip.oraclegenesis.com oraracn1-vip 192.168.0.106 oraracn2-vip.oraclegenesis.com oraracn2-vip # Single Client Access Name (SCAN) 192.168.0.110 orarac-scan.oraclegenesis.com orarac-scan
Test the network configuration
On oraracn1: ============ # ping oraracn1 --> use Ctrl+C to exit from ping # ping oraracn2 # ping oraracn1-priv # ping oraracn2-priv On oraracn2: ============ # ping oraracn1 --> use Ctrl+C to exit from ping # ping oraracn2 # ping oraracn1-priv # ping oraracn2-priv
Note: You cannot ping VIP and SCAN IP. Those IPs can only be used by an application and are not ping-able!
Configure Shared Storage
At this stage, shutdown both the virtual machines. We will allocate shared disks to RAC nodes. Open oraracn1 virtual machine settings and select storage. Click on Adds Hard Disk button under SATA controller
Choose Create new disk
Choose VDI and click Next
Select Fixed size and click Next
Allocate 100 GB of shared storage to your RAC setup
This will take some time to add shared disk to Node 1
Once disk is added, you can see the disk under Node 1 settings > storage
Now our goal is to make newly added disk on node 1 shared with node 2. For this, go to File > Virtual Media Manager
Select your shared disk > right click > choose Modify
Change the property from Normal to Shareable and click on OK
Now we are ready to share the disk with Node 2. Open oraracn2 settings > Storage > Adds hard disk icon
Do not create a new disk, Choose existing disk
Locate your shared disk at OS level. It must mostly be stored under your Node 1 location
The disk must now be shared with node 2 as well. You will see the RAC Shared Storage below
Create Shared Partitions
Now that we have 100 GB of shared storage, we need to create three partitions for the main three ASM disk groups
Create three partitions under the newly added 100 GB shared disk via node 1
On Node 1: ========== fdisk /dev/sdb n p 1 <enter> +10G ============ Frist partition is created, create second one ============ n p 1 <enter> +45G ============ Second partition is created, create third one ============= n p 1 <enter> +45G ======= All partitions are created, now save the partition table ========== w this command will save the partition table
Configure ASM & ASM Disks
As we have created 3 partitions, we must allocate each partition to single ASM disks, which will then be allocated to ASM Disk groups
Configure ASM on both nodes
On Node 1 as root user: ======================= /usr/sbin/oracleasm configure -i Default user to own the driver interface []: oracle Default group to own the driver interface []: dba Start Oracle ASM library driver on boot (y/n) [n]: y Scan for Oracle ASM disks on boot (y/n) [y]: y On Node 2 as root user: ======================= /usr/sbin/oracleasm configure -i Default user to own the driver interface []: oracle Default group to own the driver interface []: dba Start Oracle ASM library driver on boot (y/n) [n]: y Scan for Oracle ASM disks on boot (y/n) [y]: y
Load oracle-asm kernel module on both nodes. This command Creates the /etc/sysconfig/oracleasm configuration file, creates the /dev/oracleasm mount point and mounts the ASMLib driver file system
On Node 1 as root user: ======================= /usr/sbin/oracleasm init On Node 2 as root user: ======================= /usr/sbin/oracleasm init
Create ASM disks – Any one Node
On Node 1 as root user: ======================= oracleasm createdisk CRS1 /dev/sdb5 oracleasm createdisk DATA1 /dev/sdb6 oracleasm createdisk FRA1 /dev/sdb7
Run below commands on both nodes to scan and list ASM disks
oracleasm scandisks oracleasm listdisks
Oracle Grid Installation
Oracle grid software is the one which binds multiple independent servers into a cluster. You can configure ASM only when Oracle grid is installed because ASM is bundled with Oracle Grid Software.
Configure NTP – All Nodes
On Node 1 and Node 2: ===================== # yum install ntp # service ntpd start # chkconfig ntpd on # vi /etc/sysconfig/ntpd -> Open file delete all and put below line OPTIONS="-x -u ntp:ntp -p /var/run/ntpd.pid -g" # service ntpd stop # ntpdate 1.tw.pool.ntp.org # service ntpd start
Now its good time to clean-up the YUM repository to release some space from the disk
On both nodes as root user: =========================== # yum clean all
Set Bash Profile
Setup oracle user bash profile on node 1
# su - oracle # vi .bash_profile # Oracle Settings TMP=/tmp; export TMP TMPDIR=$TMP; export TMPDIR ORACLE_HOSTNAME=oraracn1; export ORACLE_HOSTNAME ORACLE_UNQNAME=RAC; export ORACLE_UNQNAME ORACLE_BASE=/u01/app/oracle; export ORACLE_BASE GRID_HOME=/u01/app/11.2.0/grid; export GRID_HOME DB_HOME=$ORACLE_BASE/product/11.2.0/db_1; export DB_HOME ORACLE_HOME=$DB_HOME; export ORACLE_HOME ORACLE_SID=RAC1; export ORACLE_SID ORACLE_TERM=xterm; export ORACLE_TERM BASE_PATH=/usr/sbin:$PATH; export BASE_PATH PATH=$ORACLE_HOME/bin:$BASE_PATH; export PATH LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib; export CLASSPAT H alias grid_env='. /home/oracle/grid_env' alias db_env='. /home/oracle/db_env'
Create a file to set grid variables on node 1. This file will set grid environmental variables making it easy to switch between DB home and grid home.
# vi /home/oracle/grid_env ORACLE_SID=+ASM1; export ORACLE_SID ORACLE_HOME=$GRID_HOME; export ORACLE_HOME PATH=$ORACLE_HOME/bin:$BASE_PATH; export PATH LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib; export CLASSPAT H
Create below file to set database environmental variables on Node 1
# vi /home/oracle/db_env ORACLE_SID=RAC1; export ORACLE_SID ORACLE_HOME=$DB_HOME; export ORACLE_HOME PATH=$ORACLE_HOME/bin:$BASE_PATH; export PATH LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib; export CLASSPAT H
Setup oracle user bash profile on node 2
# su - oracle # vi .bash_profile # Oracle Settings TMP=/tmp; export TMP TMPDIR=$TMP; export TMPDIR ORACLE_HOSTNAME=oraracn2; export ORACLE_HOSTNAME ORACLE_UNQNAME=RAC; export ORACLE_UNQNAME ORACLE_BASE=/u01/app/oracle; export ORACLE_BASE GRID_HOME=/u01/app/11.2.0/grid; export GRID_HOME DB_HOME=$ORACLE_BASE/product/11.2.0/db_1; export DB_HOME ORACLE_HOME=$DB_HOME; export ORACLE_HOME ORACLE_SID=RAC2; export ORACLE_SID ORACLE_TERM=xterm; export ORACLE_TERM BASE_PATH=/usr/sbin:$PATH; export BASE_PATH PATH=$ORACLE_HOME/bin:$BASE_PATH; export PATH LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib; export CLASSPAT H alias grid_env='. /home/oracle/grid_env' alias db_env='. /home/oracle/db_env'
Create a file to set grid variables on node 2. This file will set grid environmental variables making it easy to switch between DB home and grid home.
# vi /home/oracle/grid_env ORACLE_SID=+ASM2; export ORACLE_SID ORACLE_HOME=$GRID_HOME; export ORACLE_HOME PATH=$ORACLE_HOME/bin:$BASE_PATH; export PATH LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib; export CLASSPAT H
Create below file to set database environmental variables on Node 2
# vi /home/oracle/db_env ORACLE_SID=RAC2; export ORACLE_SID ORACLE_HOME=$DB_HOME; export ORACLE_HOME PATH=$ORACLE_HOME/bin:$BASE_PATH; export PATH LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib; export CLASSPAT H
At this stage, exit as oracle user and the open another terminal with new session OR switch to root user and then back to oracle user. This will make the new environmental variables in effect.
Now it’s time to test our environment variables and check if they are working fine or not. As oracle user, execute below commands on both nodes to check if environmental variables are set properly
Node 1: ------- su – oracle # . grid_env --To set grid env variables # env |grep ORA ORACLE_UNQNAME=RAC ORACLE_SID=+ASM1 ORACLE_BASE=/u01/app/oracle ORACLE_HOSTNAME=oraracn1 ORACLE_TERM=xterm ORACLE_HOME=/u01/app/11.2.0/grid # . db_env --To set DB env variables # env |grep ORA ORACLE_UNQNAME=RAC ORACLE_SID=RAC1 ORACLE_BASE=/u01/app/oracle ORACLE_HOSTNAME=oraracn1 ORACLE_TERM=xterm ORACLE_HOME=/u01/app/oracle/product/11.2.0/db_1
Node 2: ------- su – oracle # . grid_env --To set grid env variables # env |grep ORA ORACLE_UNQNAME=RAC ORACLE_SID=+ASM2 ORACLE_BASE=/u01/app/oracle ORACLE_HOSTNAME=oraracn2 ORACLE_TERM=xterm ORACLE_HOME=/u01/app/11.2.0/grid # . db_env --To set DB env variables # env |grep ORA ORACLE_UNQNAME=RAC ORACLE_SID=RAC2 ORACLE_BASE=/u01/app/oracle ORACLE_HOSTNAME=oraracn2 ORACLE_TERM=xterm ORACLE_HOME=/u01/app/oracle/product/11.2.0/db_1
Configure SSH Between Nodes
Configure SSH connectivity between the nodes as oracle user. There are two methods to setup SSH connectivity between the nodes is: Manual method and automatic method. The manual method is very long and you might encounter errors while doing it. The easy way is the automatic method via script provided by oracle.
As you have already copied the grid and database software files under /u02, unzip the grid software file. This will create a new folder name grid under /u02. Oracle provides automatic ssh setup script
On node 1 as oracle user: ========================= cd /u02 unzip <grid_software_file_name> cd grid/sshsetup ./sshUserSetup.sh -user oracle -hosts "oraracn1 oraracn2" -noPromptPassphrase -confir m -advanced
Run cluvfy utility
Oracle provide a pre-installation utility which is called as Cluster Verification Utility (CLUVFY). This utility performs cluster verification before, during and after install of grid software. Let us run CLUVFY utility to check cluster before installation of grid
# /u01/softwares/grid # ./runcluvfy.sh stage -pre crsinst -n oraracn1,oraracn2 -fixup -verbose
Below is the sample output:
[oracle@oraracn1 grid]$ ./runcluvfy.sh stage -pre crsinst -n oraracn1,oraracn2 -fixup -verbose Performing pre-checks for cluster services setup Checking node reachability... Check: Node reachability from node "oraracn1" Destination Node Reachable? ------------------------------------ ------------------------ oraracn2 yes oraracn1 yes Result: Node reachability check passed from node "oraracn1" Checking user equivalence... Check: User equivalence for user "oracle" Node Name Comment ------------------------------------ ------------------------ oraracn1 passed oraracn2 passed Result: User equivalence check passed for user "oracle" Checking node connectivity... Checking hosts config file... Node Name Status Comment ------------ ------------------------ ------------------------ oraracn1 passed oraracn2 passed Verification of the hosts config file successful Interface information for node "oraracn1" Name IP Address Subnet Gateway Def. Gateway HW Address MTU ------ --------------- --------------- --------------- --------------- ---------------- ------ eth0 192.168.0.151 192.168.0.0 0.0.0.0 192.168.0.1 00:0C:29:B6:0 F:87 1500 eth1 10.10.10.151 10.10.10.0 0.0.0.0 192.168.0.1 00:0C:29:B6:0 F:91 1500 Interface information for node "oraracn2" Name IP Address Subnet Gateway Def. Gateway HW Address MTU ------ --------------- --------------- --------------- --------------- ---------------- ------ eth0 192.168.0.152 192.168.0.0 0.0.0.0 192.168.0.1 00:0C:29:81:9 1:FA 1500 eth1 10.10.10.152 10.10.10.0 0.0.0.0 192.168.0.1 00:0C:29:81:9 1:04 1500 Check: Node connectivity of subnet "192.168.0.0" Source Destination Connected? ------------------------------ ------------------------------ --------- oraracn1:eth0 oraracn2:eth0 yes Result: Node connectivity passed for subnet "192.168.0.0" with node(s) oraracn1,orara cn2 Check: TCP connectivity of subnet "192.168.0.0" Source Destination Connected? -------------------- ------------------------ -------- oraracn1:192.168.0.151 oraracn2:192.168.0.152 passed Result: TCP connectivity check passed for subnet "192.168.0.0" Check: Node connectivity of subnet "10.10.10.0" Source Destination Connected? ------------------------------ ------------------- ----------- oraracn1:eth1 oraracn2:eth1 yes Result: Node connectivity passed for subnet "10.10.10.0" with node(s) oraracn1,orarac n2 Check: TCP connectivity of subnet "10.10.10.0" Source Destination Connected? ------------------------------ ------------- ---------- oraracn1:10.10.10.151 oraracn2:10.10.10.152 passed Result: TCP connectivity check passed for subnet "10.10.10.0" Interfaces found on subnet "192.168.0.0" that are likely candidates for VIP are: oraracn1 eth0:192.168.0.151 oraracn2 eth0:192.168.0.152 Interfaces found on subnet "10.10.10.0" that are likely candidates for a private inte rconnect are: oraracn1 eth1:10.10.10.151 oraracn2 eth1:10.10.10.152 Result: Node connectivity check passed Check: Total memory Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 2.44GB (2559180.0KB) 1.5GB (1572864.0KB) passed oraracn2 2.44GB (2559180.0KB) 1.5GB (1572864.0KB) passed Result: Total memory check passed Check: Available memory Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 2.11GB (2211952.0KB) 50MB (51200.0KB) passed oraracn2 2.19GB (2294412.0KB) 50MB (51200.0KB) passed Result: Available memory check passed Check: Swap space Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 4.88GB (5116692.0KB) 2.44GB (2559180.0KB) passed oraracn2 4.88GB (5116692.0KB) 2.44GB (2559180.0KB) passed Result: Swap space check passed Check: Free disk space for "oraracn1:/tmp" Path Node Name Mount point Available Required Comment ---------------- ------------ ------------ ------------ ------------ -------- /tmp oraracn1 / 4.94GB 1GB passed Result: Free disk space check passed for "oraracn1:/tmp" Check: Free disk space for "oraracn2:/tmp" Path Node Name Mount point Available Required Comment ---------------- ------------ ------------ ------------ ------------ --------- /tmp oraracn2 / 4.71GB 1GB passed Result: Free disk space check passed for "oraracn2:/tmp" Check: User existence for "oracle" Node Name Status Comment ------------ ------------------------ ------------------------ oraracn1 exists passed oraracn2 exists passed Result: User existence check passed for "oracle" Check: Group existence for "oinstall" Node Name Status Comment ------------ ------------------------ ------------------------ oraracn1 exists passed oraracn2 exists passed Result: Group existence check passed for "oinstall" Check: Group existence for "dba" Node Name Status Comment ------------ ------------------------ ------------------------ oraracn1 exists passed oraracn2 exists passed Result: Group existence check passed for "dba" Check: Membership of user "oracle" in group "oinstall" [as Primary] Node Name User Exists Group Exists User in Group Primary Comment ----------- --------- --------- ------------ ------------ ------- oraracn1 yes yes yes yes passed oraracn2 yes yes yes yes passed Result: Membership check for user "oracle" in group "oinstall" [as Primary] passed Check: Membership of user "oracle" in group "dba" Node Name User Exists Group Exists User in Group Comment ---------------- ------------ ------------ ------------ ---------------- oraracn1 yes yes yes passed oraracn2 yes yes yes passed Result: Membership check for user "oracle" in group "dba" passed Check: Run level Node Name run level Required Comment ------------ ----------- ----------- ----------- oraracn1 5 3,5 passed oraracn2 5 3,5 passed Result: Run level check passed Check: Hard limits for "maximum open file descriptors" Node Name Type Available Required Comment ---------------- ------------ ------------ ------------ ---------------- oraracn1 hard 131072 65536 passed oraracn2 hard 131072 65536 passed Result: Hard limits check passed for "maximum open file descriptors" Check: Soft limits for "maximum open file descriptors" Node Name Type Available Required Comment ---------------- ------------ ------------ ------------ ---------------- oraracn1 soft 131072 1024 passed oraracn2 soft 131072 1024 passed Result: Soft limits check passed for "maximum open file descriptors" Check: Hard limits for "maximum user processes" Node Name Type Available Required Comment ---------------- ------------ ------------ ------------ ---------------- oraracn1 hard 131072 16384 passed oraracn2 hard 131072 16384 passed Result: Hard limits check passed for "maximum user processes" Check: Soft limits for "maximum user processes" Node Name Type Available Required Comment ---------------- ------------ ------------ ------------ ---------------- oraracn1 soft 131072 2047 passed oraracn2 soft 131072 2047 passed Result: Soft limits check passed for "maximum user processes" Check: System architecture Node Name Available Required Comment ----------- ------------------------ ------------------------ ---------- oraracn1 x86_64 x86_64 passed oraracn2 x86_64 x86_64 passed Result: System architecture check passed Check: Kernel version Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 2.6.18-164.el5 2.6.18 passed oraracn2 2.6.18-164.el5 2.6.18 passed Result: Kernel version check passed Check: Kernel parameter for "semmsl" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 250 250 passed oraracn2 250 250 passed Result: Kernel parameter check passed for "semmsl" Check: Kernel parameter for "semmns" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 32000 32000 passed oraracn2 32000 32000 passed Result: Kernel parameter check passed for "semmns" Check: Kernel parameter for "semopm" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 100 100 passed oraracn2 100 100 passed Result: Kernel parameter check passed for "semopm" Check: Kernel parameter for "semmni" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 142 128 passed oraracn2 142 128 passed Result: Kernel parameter check passed for "semmni" Check: Kernel parameter for "shmmax" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 4398046511104 536870912 passed oraracn2 4398046511104 536870912 passed Result: Kernel parameter check passed for "shmmax" Check: Kernel parameter for "shmmni" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 4096 4096 passed oraracn2 4096 4096 passed Result: Kernel parameter check passed for "shmmni" Check: Kernel parameter for "shmall" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 4294967296 2097152 passed oraracn2 4294967296 2097152 passed Result: Kernel parameter check passed for "shmall" Check: Kernel parameter for "file-max" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 6815744 6815744 passed oraracn2 6815744 6815744 passed Result: Kernel parameter check passed for "file-max" Check: Kernel parameter for "ip_local_port_range" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 between 9000 & 65500 between 9000 & 65500 passed oraracn2 between 9000 & 65500 between 9000 & 65500 passed Result: Kernel parameter check passed for "ip_local_port_range" Check: Kernel parameter for "rmem_default" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 262144 262144 passed oraracn2 262144 262144 passed Result: Kernel parameter check passed for "rmem_default" Check: Kernel parameter for "rmem_max" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 4194304 4194304 passed oraracn2 4194304 4194304 passed Result: Kernel parameter check passed for "rmem_max" Check: Kernel parameter for "wmem_default" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 262144 262144 passed oraracn2 262144 262144 passed Result: Kernel parameter check passed for "wmem_default" Check: Kernel parameter for "wmem_max" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 1048576 1048576 passed oraracn2 1048576 1048576 passed Result: Kernel parameter check passed for "wmem_max" Check: Kernel parameter for "aio-max-nr" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 3145728 1048576 passed oraracn2 3145728 1048576 passed Result: Kernel parameter check passed for "aio-max-nr" Check: Package existence for "make-3.81" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 make-3.81-3.el5 make-3.81 passed oraracn2 make-3.81-3.el5 make-3.81 passed Result: Package existence check passed for "make-3.81" Check: Package existence for "binutils-2.17.50.0.6" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 binutils-2.17.50.0.6-12.el5 binutils-2.17.50.0.6 passed oraracn2 binutils-2.17.50.0.6-12.el5 binutils-2.17.50.0.6 passed Result: Package existence check passed for "binutils-2.17.50.0.6" Check: Package existence for "gcc-4.1.2" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 gcc-4.1.2-46.el5 gcc-4.1.2 passed oraracn2 gcc-4.1.2-46.el5 gcc-4.1.2 passed Result: Package existence check passed for "gcc-4.1.2" Check: Package existence for "libaio-0.3.106 (i386)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 libaio-0.3.106-5 (i386) libaio-0.3.106 (i386) passed oraracn2 libaio-0.3.106-5 (i386) libaio-0.3.106 (i386) passed Result: Package existence check passed for "libaio-0.3.106 (i386)" Check: Package existence for "libaio-0.3.106 (x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 libaio-0.3.106-5 (x86_64) libaio-0.3.106 (x86_64) passed oraracn2 libaio-0.3.106-5 (x86_64) libaio-0.3.106 (x86_64) passed Result: Package existence check passed for "libaio-0.3.106 (x86_64)" Check: Package existence for "glibc-2.5-24 (i686)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 glibc-2.5-42 (i686) glibc-2.5-24 (i686) passed oraracn2 glibc-2.5-42 (i686) glibc-2.5-24 (i686) passed Result: Package existence check passed for "glibc-2.5-24 (i686)" Check: Package existence for "glibc-2.5-24 (x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 glibc-2.5-42 (x86_64) glibc-2.5-24 (x86_64) passed oraracn2 glibc-2.5-42 (x86_64) glibc-2.5-24 (x86_64) passed Result: Package existence check passed for "glibc-2.5-24 (x86_64)" Check: Package existence for "compat-libstdc++-33-3.2.3 (i386)" Node Name Available Required Comment ------------ ------------------------ ----------------------- ------- oraracn1 compat-libstdc++-33-3.2.3-61 (i386) compat-libstdc++-33-3.2.3 (i386) passed oraracn2 compat-libstdc++-33-3.2.3-61 (i386) compat-libstdc++-33-3.2.3 (i386) passed Result: Package existence check passed for "compat-libstdc++-33-3.2.3 (i386)" Check: Package existence for "compat-libstdc++-33-3.2.3 (x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 compat-libstdc++-33-3.2.3-61 (x86_64) compat-libstdc++-33-3.2.3 (x86 _64) passed oraracn2 compat-libstdc++-33-3.2.3-61 (x86_64) compat-libstdc++-33-3.2.3 (x86 _64) passed Result: Package existence check passed for "compat-libstdc++-33-3.2.3 (x86_64)" Check: Package existence for "elfutils-libelf-0.125 (x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 elfutils-libelf-0.137-3.el5 (x86_64) elfutils-libelf-0.125 (x86_64) passed oraracn2 elfutils-libelf-0.137-3.el5 (x86_64) elfutils-libelf-0.125 (x86_64) passed Result: Package existence check passed for "elfutils-libelf-0.125 (x86_64)" Check: Package existence for "elfutils-libelf-devel-0.125" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 elfutils-libelf-devel-0.137-3.el5 elfutils-libelf-devel-0.125 passed oraracn2 elfutils-libelf-devel-0.137-3.el5 elfutils-libelf-devel-0.125 passed Result: Package existence check passed for "elfutils-libelf-devel-0.125" Check: Package existence for "glibc-common-2.5" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 glibc-common-2.5-42 glibc-common-2.5 passed oraracn2 glibc-common-2.5-42 glibc-common-2.5 passed Result: Package existence check passed for "glibc-common-2.5" Check: Package existence for "glibc-devel-2.5 (i386)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 glibc-devel-2.5-42 (i386) glibc-devel-2.5 (i386) passed oraracn2 glibc-devel-2.5-42 (i386) glibc-devel-2.5 (i386) passed Result: Package existence check passed for "glibc-devel-2.5 (i386)" Check: Package existence for "glibc-devel-2.5 (x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 glibc-devel-2.5-42 (x86_64) glibc-devel-2.5 (x86_64) passed oraracn2 glibc-devel-2.5-42 (x86_64) glibc-devel-2.5 (x86_64) passed Result: Package existence check passed for "glibc-devel-2.5 (x86_64)" Check: Package existence for "glibc-headers-2.5" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 glibc-headers-2.5-42 glibc-headers-2.5 passed oraracn2 glibc-headers-2.5-42 glibc-headers-2.5 passed Result: Package existence check passed for "glibc-headers-2.5" Check: Package existence for "gcc-c++-4.1.2" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 gcc-c++-4.1.2-46.el5 gcc-c++-4.1.2 passed oraracn2 gcc-c++-4.1.2-46.el5 gcc-c++-4.1.2 passed Result: Package existence check passed for "gcc-c++-4.1.2" Check: Package existence for "libaio-devel-0.3.106 (i386)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 libaio-devel-0.3.106-5 (i386) libaio-devel-0.3.106 (i386) passed oraracn2 libaio-devel-0.3.106-5 (i386) libaio-devel-0.3.106 (i386) passed Result: Package existence check passed for "libaio-devel-0.3.106 (i386)" Check: Package existence for "libaio-devel-0.3.106 (x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 libaio-devel-0.3.106-5 (x86_64) libaio-devel-0.3.106 (x86_64) passed oraracn2 libaio-devel-0.3.106-5 (x86_64) libaio-devel-0.3.106 (x86_64) passed Result: Package existence check passed for "libaio-devel-0.3.106 (x86_64)" Check: Package existence for "libgcc-4.1.2 (i386)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 libgcc-4.1.2-46.el5 (i386) libgcc-4.1.2 (i386) passed oraracn2 libgcc-4.1.2-46.el5 (i386) libgcc-4.1.2 (i386) passed Result: Package existence check passed for "libgcc-4.1.2 (i386)" Check: Package existence for "libgcc-4.1.2 (x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 libgcc-4.1.2-46.el5 (x86_64) libgcc-4.1.2 (x86_64) passed oraracn2 libgcc-4.1.2-46.el5 (x86_64) libgcc-4.1.2 (x86_64) passed Result: Package existence check passed for "libgcc-4.1.2 (x86_64)" Check: Package existence for "libstdc++-4.1.2 (i386)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 libstdc++-4.1.2-46.el5 (i386) libstdc++-4.1.2 (i386) passed oraracn2 libstdc++-4.1.2-46.el5 (i386) libstdc++-4.1.2 (i386) passed Result: Package existence check passed for "libstdc++-4.1.2 (i386)" Check: Package existence for "libstdc++-4.1.2 (x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 libstdc++-4.1.2-46.el5 (x86_64) libstdc++-4.1.2 (x86_64) passed oraracn2 libstdc++-4.1.2-46.el5 (x86_64) libstdc++-4.1.2 (x86_64) passed Result: Package existence check passed for "libstdc++-4.1.2 (x86_64)" Check: Package existence for "libstdc++-devel-4.1.2 (x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 libstdc++-devel-4.1.2-46.el5 (x86_64) libstdc++-devel-4.1.2 (x86_64) passed oraracn2 libstdc++-devel-4.1.2-46.el5 (x86_64) libstdc++-devel-4.1.2 (x86_64) passed Result: Package existence check passed for "libstdc++-devel-4.1.2 (x86_64)" Check: Package existence for "sysstat-7.0.2" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 sysstat-7.0.2-13.0.1.el5 sysstat-7.0.2 passed oraracn2 sysstat-7.0.2-13.0.1.el5 sysstat-7.0.2 passed Result: Package existence check passed for "sysstat-7.0.2" Check: Package existence for "unixODBC-2.2.11 (i386)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 unixODBC-2.2.11-10.el5 (i386) unixODBC-2.2.11 (i386) passed oraracn2 unixODBC-2.2.11-10.el5 (i386) unixODBC-2.2.11 (i386) passed Result: Package existence check passed for "unixODBC-2.2.11 (i386)" Check: Package existence for "unixODBC-2.2.11 (x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 unixODBC-2.2.11-10.el5 (x86_64) unixODBC-2.2.11 (x86_64) passed oraracn2 unixODBC-2.2.11-10.el5 (x86_64) unixODBC-2.2.11 (x86_64) passed Result: Package existence check passed for "unixODBC-2.2.11 (x86_64)" Check: Package existence for "unixODBC-devel-2.2.11 (i386)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 unixODBC-devel-2.2.11-10.el5 (i386) unixODBC-devel-2.2.11 (i386) passed oraracn2 unixODBC-devel-2.2.11-10.el5 (i386) unixODBC-devel-2.2.11 (i386) passed Result: Package existence check passed for "unixODBC-devel-2.2.11 (i386)" Check: Package existence for "unixODBC-devel-2.2.11 (x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 unixODBC-devel-2.2.11-10.el5 (x86_64) unixODBC-devel-2.2.11 (x86_64) passed oraracn2 unixODBC-devel-2.2.11-10.el5 (x86_64) unixODBC-devel-2.2.11 (x86_64) passed Result: Package existence check passed for "unixODBC-devel-2.2.11 (x86_64)" Check: Package existence for "ksh-20060214" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 ksh-20080202-14.el5 ksh-20060214 passed oraracn2 ksh-20080202-14.el5 ksh-20060214 passed Result: Package existence check passed for "ksh-20060214" Checking for multiple users with UID value 0 Result: Check for multiple users with UID value 0 passed Check: Current group ID Result: Current group ID check passed Checking Core file name pattern consistency... Core file name pattern consistency check passed. Checking to make sure user "oracle" is not in "root" group Node Name Status Comment ------------ ------------------------ ------------------------ oraracn1 does not exist passed oraracn2 does not exist passed Result: User "oracle" is not part of "root" group. Check passed Check default user file creation mask Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- oraracn1 0022 0022 passed oraracn2 0022 0022 passed Result: Default user file creation mask check passed Starting Clock synchronization checks using Network Time Protocol(NTP)... NTP Configuration file check started... Network Time Protocol(NTP) configuration file not found on any of the nodes. Oracle C luster Time Synchronization Service(CTSS) can be used instead of NTP for time synchro nization on the cluster nodes Result: Clock synchronization check using Network Time Protocol(NTP) passed Pre-check for cluster services setup was successful.
Note: You must see Pre-check for cluster services setup was successful at the end of the cluvfy output.
Install Grid Software
The following steps will guide you to install Oracle Grid software with Oracle ASM binaries. As oracle user, start the grid runInstaller
On Node 1: ========== su – oracle # cd /u02/grid/ # ./runInstaller
Note: sometimes there might be display variable error. issues below command as root user to enable GUI
# xhost +
Under Select Installation Option, choose Install and Configure Grid Infrastructure for a Cluster and Click Next
Select Advanced Installation and click Next
Select Advanced Installation and click Next
Input below details and click Next
Below screen will list out the nodes which you want to install and configure as part of Grid
Click on Add and enter details of Node 2. Click on OK
In case you have more than two nodes, Add each node by clicking on Add. Click Next to proceed
Specify network interface by selecting public and private interfaces. Click Next
Select storage option as Automatic Storage Management (ASM). Click Next
You need to assign a disk where the grid installer will keep OCR and Voting disks. Click on Change Discovery Path, type /dev/oracleasm/disks/* and click OK. Under Add Disks, you must see the three disks which we have created as part of Shard Storage.
Before you proceed, sometimes you might not see the lists of disks. In such cases, follow below:
1. Check if /dev/sdb1, /dev/sdc1, /dev/sdd1 had ownership as oracle user. If not then issue below commands as root user on both nodes:
# chown oracle:oinstall /dev/sdb1 # chown oracle:oinstall /dev/sdc1 # chown oracle:oinstall /dev/sdd1
2. Also give full permissions on above disks
# chmod 777 /dev/sdb1 # chmod 777 /dev/sdc1 # chmod 777 /dev/sdd1
Again go to Change Discovery Path, type /dev/oracleasm/disks/* and click OK. Now all the disks will be listed. Type Disk Group Name as CRS, Redundancy as External and under Disk Path /dev/oracleasm/disks/CRSVOL1
Now provide password for sys user and click Next
It will prompt you for password standard, just click on Yes
Select Do Not Use IPMI and click Next
Go with default operating system groups and click Next. If prompted with notification, just click on Yes
Provide Oracle Base and Software Installation Location. Click on Next
Go with default inventory location. Click Next
The run Installer will perform pre-requsites test again. If any step is failed, you must fix it before proceeding ahead
The final screen displays installation summary. Important things to note here:
Oracle Base Location
Oracle Home – This should actually be grid home
Click on Finish to start the installation
Now the setup will start installing Grid Software on both nodes
Now the installer will prompt you for two root.sh scripts to be run on each node. Execute them as root user on both nodes one by one.
Your grid installation will finish and Cluster Verification Utility Filed error will be displayed. Just ignore the error.
Oracle RAC Installation
During the installation of Oracle grid infrastructure, we configured one ASM disk group named +CRS which was used to store the Oracle clusterware files (OCR and voting disk). Now we will configure ASM disk groups for DATA and FRA
Create DATA & FRA Diskgroups
Launch asmca form the command prompt as oracle user
On node 1: ========== su – oracle asmca
ASM Configuration Assistant window will launch. From the Disk Groups tab, click on Create.
The Create Disk Group window will open and it should show two of the ASMLib volumes we created earlier
Click on Create. Give a name to your DATA disk group. This disk group will hold database files. Then click on OK
Install Oracle RAC Software – Node 1 only
Unzip the database software files under /u02. There will be two files downloaded. Unzip them as oracle user one by one
# su - oracle # cd /u02 # unzip linux.x64_11gR2_database_1of2.zip # unzip linux.x64_11gR2_database_2of2.zip
Unzipping files will create a new folder as database under same location
# ls -lrt drwxr-xr-x 8 oracle oinstall 4096 Aug 21 2009 database -rwxr-xr-x 1 oracle oinstall 1052897657 Nov 2 09:41 linux.x64_11gR2_grid.zip -rwxr-xr-x 1 oracle oinstall 1239269270 Dec 1 01:00 linux.x64_11gR2_database_1of2.zip -rwxr-xr-x 1 oracle oinstall 1111416131 Dec 1 01:01 linux.x64_11gR2_database_2of2.zip
Go to database folder and start the runInstaller
# cd database # ./runInstaller
Create RAC Database
From node 1, start dbca as oracle user
On node 1: ========== su - oracle dbca
Done! Your 2 node RAC database is ready to use!
500 views