Install > Oracle setup for the model repository > Oracle RAC support

Oracle RAC support

SA supports Oracle Real Application Clusters (RAC). Oracle RAC support requires a new installation of both Oracle and SA. Therefore, in order to enable Oracle RAC support in SA, you must first install Oracle RAC 11g, configured as described in the following sections:

Supported Oracle versions and operating systems

Support for the Model Repository is limited to certain versions of Oracle running on certain versions of operating systems. HPE strongly recommends that you also apply the latest Oracle CPU or PSU patches.

See the SA 10.60 Support and Compatibility Matrix for a list of supported Oracle versions and operating systems.

System requirements

For information on system requirements for Oracle database, see System requirements for Oracle database.

Setting up the Oracle RAC database/instances

SA supports any valid Oracle RAC configuration, such as any number of nodes, ASM or regular disks, and so on.

However, the Oracle database must be configured for using with SA. You require your Oracle database administrator's (DBA) help to configure the Oracle RAC/instances, the required initialization parameters, the required tablespaces, the opsware_admin database user, and the listener.ora and tnsnames.ora files.

Creating the database with the required initialization parameters

Perform the following tasks listed in the Non-SA-supplied Oracle software and database setup section:

  • Modifiable Kernel Parameters
  • Oracle database installation
    1. Create the database with the UTF8 database character set
    2. Set the database with TIME_ZONE to '+00:00'
    3. Create the database with the required initialization (init.ora) parameter
    4. Create the database with required tablespaces
    5. Create the database user opsware_admin
Note: Use the tnsnames.ora file and file linking requirements listed in the following sections since they differ for the Oracle RAC environment from those listed in Non-SA-supplied Oracle software and database setup.

Non-SA-supplied Oracle software and database setup describes the required database setup, Oracle initialization parameters, required tablespaces, database user opsware_admin, etc.

Note On an SA Oracle RAC DB installation, the admin should not modify the Management Gateway Properties File(s) while in operational mode. While the SA system is in operational mode, the remote Oracle DB connections are direct and do not use the RAC scan address.

Installing the Model Repository

In most production environments with Oracle RAC, you can perform the Model Repository installation from any SA server. The database server or RAC nodes in this case are considered to be remote.

The examples used in the following sections assume an SA server (rac1sa.dev.opsware.com) on which SA will be installed and a 2 node RAC configuration shown in below:

Sample RAC configurations

Identity

Host note

Name

Type

Address

Address Static or Dynamic

Resolved by

Node 1
Public

rac1pub

rac1pub

Public

192.168.173.210

Static

DNS

Node 1
Virtual

Selected by Oracle Clusterware

rac1-vip

Virtual

192.168.173.212

Static

DNS and/or host file

Node 1
Private

rac1pub

rac1prv

Private

172.16.1.100

Static

DNS, host file or none

Node 2
Public

rac2pub

rac2pub

Public

192-168-173-211

Static

DNS

Node 2
Virtual

Selected by Oracle Clusterware

rac2-vip

Virtual

192.168.173.213

Static

DNS and/or host file

Node 2
Private

rac2pub

rac2prv

Private

172.16.1.101

Static

DNS, host file or none

SCAN
vip 1

Selected by Oracle Clusterware

sa_cluster1-scan

Virtual

192.168.173.216

Static

DNS

SCAN
vip 2

Selected by Oracle Clusterware

sa_cluster1-scan

Virtual

192.168.173.217

Static

DNS

SCAN
vip 3

Selected by Oracle Clusterware

sa_cluster1-scan

Virtual

192.168.173.218

Static

DNS

Installing the Model Repository in a RACed environment

In an Oracle RAC environment, only one of the RAC nodes is used during the SA installation/upgrade process. The SA Installer connects to only one Oracle RAC instance to install/modify the Model Repository. During the regular SA operations, all RAC nodes are used.

Perform the following tasks on the SA server on which you will run the SA Installer; for example, rac1sa.dev.opsware.com.

  1. Model Repository hostname resolution

    On the server where you will run the SA Installer, ensure that the Model Repository host name truth resolves to the remote database server, not to the server on which you will be running the SA Installer:

    In /etc/hosts, enter the public IP address of one of the RAC nodes/instances. For example the /etc/hosts file on rac1sa.dev.opsware.com would have the following entry:

    192.168.173.210 truth rac1pub rac1pub.dev.opsware.com

    Note: If you have set up Oracle Clusterware, you should use the Clusterware IP address rather than a single database node IP address. For example:

    192.168.173.216 truth sa_cluster1-scan sa_cluster1-scan.dev.opsware.com

    If you have set up SCAN name, you should use the SCAN address rather than the database node IP address.
  1. Install the Oracle 11g Full Client on the SA Server
    Note: For Oracle 11.2.0.2, use the Oracle Full Client version 11.2.0.2.
    1. The SA Installer uses the Oracle Full Client to connect to the SA server and install the Model Repository. Below are sample commands for installing the Oracle full client.

      Create the database user oracle for the Oracle Full Client installation:

      root@rac1sa ~]# mkdir -p /u01/app/oracle
root@rac1sa ~]# mkdir -p /u01/app/oraInventory
root@rac1sa ~]# groupadd oinstall
root@rac1sa ~]# groupadd dba
root@rac1sa ~]# useradd -c "Oracle Client software owner" -g oinstall -G
dba -d /u01/app/oracle -s /bin/bash oracle
root@rac1sa ~]# chown -R oracle:oinstall /u01/app
root@rac1sa ~]# chmod -R 775 /u01/app
root@rac1sa ~]#passwd oracle (change oracle user password)
    2. Create the .bash_profile file.

      Considering the above example, create the .bash_profile file in /u01/app/oracle.

      Temporarily comment out ORACLE_HOME and ORACLE_PATH. You must uncomment these entries after the Oracle client installation is complete.

      Sample .bash_profile file

      # .bash_profile
# Get the aliases and functions
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
 
# User specific environment and startup programs
PATH=$PATH:$HOME/bin
export PATH
 
#SA-OracleRAC parameters begin
#unset USERNAME
export ORACLE_BASE=/u01/app/oracle
#export ORACLE_HOME=$ORACLE_BASE/product/11.2.0/client_1
#PATH=$ORACLE_HOME/bin:$ORACLE_HOME/OPatch:$PATH
export PATH
 
if [ -t ]; then
stty intr ^C
fi
 
umask 022
#SA-OracleRAC parameters end
    3. Install the Oracle Full Client.

      Install the Oracle Full Client as described in your Oracle documentation. You can create a share to access the Oracle Full Client binaries.

    4. Set up Terminals.

      You need two X window terminals to install the Oracle Full Client:

      Terminal 1: log in as root and enter the commands:

      Terminal 1> xhost +
      Terminal 2:  ssh –X oracle@<new_oracle_full_client_host>
    5. Start Oracle Full Client installation

      From Terminal 2, run the Oracle Universal Installer (OUI). The Oracle Full Client is installed at /u01/app/oracle/product/12.1.0/client_1.

    6. Run the Oracle Universal Installer to install Oracle Full Client. The directories in this example assume an Oracle 11g Full Client on Linux.

      cd /<location_of_oracle_full_client>

      ./runInstaller

    7. At the Welcome Screen, click Next.
    8. Specify the Inventory Directory and Credentials (/u01/app/oraInventory and
      /u01/app/oinstall).
    9. For Select Installation Type, choose Administrator, click Next.

    10. For ORACLE_BASE select /u01/app/oracle, click Next.

    11. The Oracle Universal Installer performs some checks. If the checks are not successful, fix the issue and re-run this step. If the checks are successful click Next.

    12. The Oracle OUI will list the products that are to be installed. Click Install.

    13. The OUI shows the progress bar while installing.

    14. On the Welcome to Oracle Net Configuration Assistant window click Next.

    15. Click Finish when the installation completes.

    16. You must run the following two configuration scripts as root after installation completes:

      /u01/app/oraInventory/orainstRoot.sh

      /u01/app/oracle/product/12.1.0/client_1/root.sh

    17. Verify that the .bash_profile file for the user oracle is correct.
    18. Uncomment $ORACLE_HOME and $ORACLE_PATH.
  2. Making changes to tnsnames.ora on an SA Server (Use tnsnames.ora-install_upgrade File)

    By default SA expects the tnsnames.ora file to be located in /var/opt/oracle.

    1. Login as root on the SA server from which the installer will be run.
    2. Enter the command:

      mkdir -p /var/opt/oracle

    3. Copy tnsnames.ora from the remote database server to the directory you created above.

      For the RAC environment, copy tnsnames.ora from RAC Node 1 (for example, rac1pub.dev.opsware.com).

      To accommodate the remote Model Repository installation process, two sets of tnsnames.ora files are required on the SA server.

      • tnsnames.ora-install_upgrade – this copy of tnsnames.ora is used during SA installation/upgrade. The file can be renamed.
      • tnsnames.ora_install_upgrade – this copy of tnsnames.ora is used during normal SA operation. The file can be renamed.

      You can use softlinks to point tnsnames.ora to either tnsnames.ora-install_upgrade or tnsnames.ora_install_upgrade. For example:

      ln –s tnsnames.ora-install_upgrade tnsnames.ora

      tnsnames.ora_install_upgrade sample file

      # tnsnames.ora Network Configuration File: /u01/app/oracle/product/12.1.0/db_1/network/admin/tnsnames.ora
      # Generated by Oracle configuration tools.
      RAC1SA_TRUTH =
      (DESCRIPTION =
      (ADDRESS = (PROTOCOL = TCP)(HOST = rac1pub.dev.opsware.com)
      (PORT = 1521))
      (CONNECT_DATA =

      (SERVER = DEDICATED)

      (SID = truth)

      )
      )
      RAC2SA_TRUTH=(DESCRIPTION=(ADDRESS=(HOST=192.168.173.214)(PORT=20002)(PROTOCOL=tcp))(CONNECT_DATA=(SERVICE_NAME=truth)))

Testing the connection from the SA host to the database

Before starting the Model Repository installation/upgrade, you can perform the following tests to verify that your tnsnames.ora file is configured correctly and if the SA Installer can connect to the database.

  1. Verify that the SA server's /var/opt/oracle/tnsnames.ora file is configured correctly as described in Making Changes to tnsnames.ora on an SA Server (Use tnsnames.ora-install_upgrade File).
  2. On the SA server:
    1. Log in as oracle or root or su – twist/spin – if these users exist.
    2. export ORACLE_HOME=/u01/app/oracle/product/12.1.0/client_1 (or where you installed the Oracle Full Client)
    3. export LD_LIBRARY_PATH=$ORACLE_HOME/lib
    4. export TNS_ADMIN=/var/opt/oracle
    5. set $PATH $ORACLE_HOME/bin path
    6. sqlplus sys/password@RAC1SA_TRUTH as sysdba;

      where rac1sa_truth is the service_name or entry from the tnsnames.ora file

    7. connect opsware_admin/<password>@RAC1SA_truth

      If you are able to log on to the database, then all files are configured correctly.

SA Installer Core Definition File (CDF)

The installer should be run in ‘Expert’ mode so that several parameter values can be specified.

You can now start the installation of the SA Model Repository. Ensure that you have the correct parameters values for the installation interview or that you have a previous Core Definition File (CDF).

  • %db.sid: truth1 (Oracle SID of the instance where SA installer is going to connect to.)
  • %db.orahome: /u01/app/oracle/product/12.1.0/client_1(Oracle client home)
  • %db.port: 1521(Oracle listener port)
  • %db.host: 192.168.173.210 (server where Oracle RDBMS is installed)
  • %truth.servicename: rac1sa_truth (value of service name from tnsnames.ora file)

You can now install the SA Core as described in the SA Core installation.

Modify vault.conf SA Installer Core Definition File (CDF)

During the installation process, the vault might not re-start. Change the vault.conf to include the RACed environment connect string. Refer to vault.conf File changes.

Post SA installation process

After you install the SA Core, perform the following tasks in order to use all the nodes in the Oracle RAC environment.

Making changes to tnsnames.ora on the SA Server (Use tnsnames.ora_install_upgrade file)

After SA Core installation is complete, the tnsnames.ora file should point/link to the tnsnames.ora_install_upgrade file.

In an Oracle RAC environment, only one of the RAC nodes or instances is used during the installation/upgrade process. The SA Installer connects to only one Oracle instance to modify the Model Repository. During normal SA operations, all the RAC nodes are used.

To accommodate the remote database installation process, two sets of tnsnames.ora files are required on the SA server.

  • tnsnames.ora-install_upgrade – this copy of tnsnames.ora is used during SA installation/upgrade. You can rename the file.
  • tnsnames.ora_install_upgrade – this copy of tnames.ora is used during normal SA operation. You can rename the file.

    You can use softlinks to point tnsnames.ora to either tnsnames.ora-install_upgrade or tnsnames.ora_install_upgrade:

    ln –s tnsnames.ora_install_upgrade tnsnames.ora

tnsnames.ora_install_upgrade sample file

Note: If you have set up Oracle Clusterware, you should use the Clusterware IP address rather than a single database node IP address. If you have set up SCAN name, you should use the SCAN address rather than the database node IP address.

Make a note of the text that is in bold letters. This tnsnames.ora file is used during normal SA operation and contains the RAC parameters.

tnsnames.ora_install_upgrade sample file - with Clusterware setup

If you have set up Oracle Clusterware, use the following:

#This entry is for connecting to RAC virtual machines. This entry is used by SA during operation of SA.

RAC1SA_TRUTH =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = sa_cluster1-scan.dev.opsware.com)(PORT = 1521))
(LOAD_BALANCE = yes)
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = truth1)
(FAILOVER_MODE =)
(TYPE = SELECT)
(METHOD = Preconnect)
(RETRIES = 180)
(DELAY = 5))
)
)
#This entry is for connecting to node2 via service_name. This is for DBA convenience. This is not used by SA.
RAC2SA_TRUTH =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = rac2pub.dev.opsware.com)(PORT = 1521))
(CONNECT_DATA =
(UR=A)
(SERVER = DEDICATED)
(SERVICE_NAME = truth2)
)
)

#This entry is for connecting to node1 via service_name. This is for DBA convenience. This is not used by SA.

TRUTH1 =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = rac1pub.dev.opsware.com)(PORT = 1521))
(CONNECT_DATA =
 
(SERVER = DEDICATED) 
(SERVICE_NAME = truth)
)
)

During installation, the SA Installer adds an SA Gateway entry into tnsnames.ora file (linked to tnsnames.ora.install-upgrade) on the Primary SA Core.

After installation completes, copy that entry into the tnsname.ora.operational file. If this entry is not present in the tnsname.ora.operational file, Multimaster Mesh transactions will not flow. The following is a sample gateway entry from tnsnames.ora:

RAC2SA_TRUTH=(DESCRIPTION=(ADDRESS=(HOST=192.168.173.214) 

(PORT=20002)
(PROTOCOL=tcp))
(CONNECT_DATA=(SERVICE_NAME=truth)))

tnsnames.ora_install_upgrade sample file - without Clusterware setup

If you have not set up Oracle Clusterware, use the following:

#This entry is for connecting to RAC virtual machines.

RAC1SA_TRUTH =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = rac1-vip.dev.opsware.com)(PORT = 1521))
(ADDRESS = (PROTOCOL = TCP)(HOST = rac2-vip.dev.opsware.com)(PORT = 1521))
(LOAD_BALANCE = yes)
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = truth)
(FAILOVER_MODE =)
(TYPE = SELECT)
(METHOD = Preconnect)
(RETRIES = 180)
(DELAY = 5))

)

)

LISTENERS_TRUTH =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = rac1-vip.dev.opsware.com)(PORT = 1521))
(ADDRESS = (PROTOCOL = TCP)(HOST = rac2-vip.dev.opsware.com)(PORT = 1521))

)

#This entry is for connecting to node2 via service_name. This entry is optional. This is for DBA convenience. This is not used by SA.

RAC2SA_TRUTH2 =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = rac2-vip.dev.opsware.com)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = truth)
(INSTANCE_NAME = truth2)
)
)
LISTENER_TRUTH2 =

(ADDRESS = (PROTOCOL = TCP)(HOST = rac2-vip.dev.opsware.com)(PORT = 1521))

#This entry is for connecting to node1 using service_name. This entry is optional. This is for DBA convenience. This is not used by SA.

TRUTH1 =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = rac1-vip.dev.opsware.com)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = truth)
(INSTANCE_NAME = truth1)
)
)
LISTENER_TRUTH1 =
(ADDRESS = (PROTOCOL = TCP)(HOST = rac1-vip.dev.opsware.com)(PORT = 1521))

During installation, the SA Installer adds an SA Gateway entry into tnsnames.ora file (linked to tnsnames.ora.install-upgrade) on the Primary SA Core.

After installation completes, copy that entry into the tnsname.ora.operational file. If this entry is not present in the tnsname.ora.operational file, Multimaster Mesh transactions will not flow. The following is a sample gateway entry from tnsnames.ora:

RAC2SA_TRUTH=(DESCRIPTION=(ADDRESS=(HOST=192.168.173.214)
(PORT=20002) (PROTOCOL=tcp))(CONNECT_DATA=(SERVICE_NAME=truth)))

Use softlinks to link the file to tnsnames.ora file after SA installation is complete and you are ready to start SA in operational mode.

vault.conf file changes

Note: If you have set up Oracle Clusterware, you should use the Clusterware IP address rather than a single database node IP address. If you have set up SCAN name, you should use the SCAN address rather than the database node IP address.

In an Oracle RAC environment, the vault.conf file must be modified after SA installation is complete. Modify /etc/opt/opsware/vault/vault.conf to specify the complete tnsnames.ora definition instead of the SID. For example:

  • If you have set up Oracle Clusterware, use the following:

    Before:

    db.sid: truth

    After:

    #truth.sid: truth1
    truth.sid: (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)
    (HOST = sa_cluster1-scan)(PORT = 1521))
    (LOAD_BALANCE = yes)
    (CONNECT_DATA =(SERVER = DEDICATED)
    (SERVICE_NAME = truth)
    (FAILOVER_MODE = (TYPE = SELECT)
    (METHOD = Preconnect)(RETRIES = 180)(DELAY = 5))))
  • If Oracle Clusterware is not set up, use the following: 
    #truth.sid: truth1
    truth.sid:(DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)
    (HOST =rac1-vip.dev.opsware.com)(PORT = 1521)) (ADDRESS = (PROTOCOL = TCP)
    (HOST = rac2-vip.dev.opsware.com)(PORT = 1521)) 
    (LOAD_BALANCE = yes)
    (CONNECT_DATA = (SERVER = DEDICATED) 
    (SERVICE_NAME = truth)
    (FAILOVER_MODE = (TYPE = SELECT) 
    (METHOD = Preconnect) (RETRIES = 180)(DELAY = 5))))
  • Also, ensure that these values are correct:

    truth.port: 1521

    truth.host: 192.168.173.210 (database server IP)
    truth.servicename: rac1sa_truth (tnsnames.ora enter)
  • Restart the vaultdaemon:

    /etc/init.d/opsware-sas restart vaultdaemon

da.conf file changes

As of SA 9.10 and later, the Application Deployment Manager reads database connection information from the tnsnames.ora file.

In SA 9.10 and 9.1x, the default was SID =Truth unless changed by the user; for example, in
/etc/opt/opsware/da/da.conf:

truth.sid=truth1 (this is the Oracle SID of the instance on RAC node)

opsware_start.config file changes

This file is located in:

/opt/opsware/oi_util/startup/opsware_start.config

Note: If you have set up Oracle Clusterware, you should use the Clusterware IP address rather than a single database node IP address. If you have set up SCAN name, you should use the SCAN address rather than the database node IP address.
  • TRUTH_HOST="192.168.173.210" - If Clusterware is not set up, then set the TRUTH_HOST value to one of the node's hostnames or public IPs.
  • TRUTH_HOST="sa_cluster1-scan.dev.opsware.com" - If Clusterware is set up, then set the TRUTH_HOST value to the Clusterware scan name.

Setting up a Secondary SA Core in an Oracle RACed environment

During the interview process, the installer asks for the secondary cores’ database host information. Enter the IP or host name of the secondary cores single RACed node. During the install process, the installer connects to the database via a single node only.

Upgrading the Model Repository in a RACed environment

To upgrade the Model Repository in an Oracle RAC environment, follow the same procedure as Installing the Model Repository. If you are doing a remote database installation, ensure that you modify the tnsnames.ora file on the server on which the SA Installer is run. HPE recommends that you test the connection as described in Testing the connection from the SA host to the database.

Setting the Oracle database server OS time zone to a non-UTC

The Oracle Standard, Enterprise or RACed database servers can be set to the non-UTC time zone. For more information, see KM01925296.

Send Help Center feedback