High Availability Mode JMX Methods

Replace the Writer Server

In the JMX Console, you can invoke the High Availability Services > suggestNewWriterServer method where you can suggest which server (serverID) should replace the Writer server.

High Availability Cluster Authentication

To enable cluster authentication:

1. In UCMDB, go to Administration > Infrastructure Settings Manager.
2. Find the setting Enable joining High Availability cluster authentication and set it to true.

3. Provide a single server authentication keystore (certificate + private and public keys) in JKS format. This keystore will be placed on all the servers and used for authenticating when connecting to a high availability cluster.

   Place the keystore in the following location: <UCMDB_install_dir>\conf\security and name it cluster.authentication.keystore.

   Note The UCMDB comes with this keystore pre-configured out-of-the-box. This keystore is the same for all clean UCMDB installations, and thus not secure. If you wish to securely authenticate join requests, delete this file and create a new one.

4. Generate a cluster authentication keystore as follows:

   1. From C:\UCMDB\UCMDBServer\bin\jre\bin, run the following command:

      keytool -genkey -alias hpcert –keystore <UCMDB_install_dir>\conf\security\cluster.authentication.keystore -keyalg RSA

      The console dialog box opens and asks you for a new keystore password.

   2. The default password is hppass. If you want to use a different password, update the server by running the following JMX method: UCMDB:service=High Availability Services: changeClusterAuthenticationKeystorePassword

   3. In the console dialog box, answer the question What is your first and last name? by entering the name of the cluster.
   4. Enter the other parameters according to your organization's details.

   5. Enter a key password. The key password must be the same as the keystore password.

      A JKS keystore is created in <UCMDB_install_dir>\conf\security\cluster.authentication.keystore

5. Replace the old <UCMDB_install_dir>\conf\security\cluster.authentication.keystore on all the servers in the cluster with the new keystore.

6. Restart all the servers in the cluster.

Changing the Key in the key.bin

In a High Availability environment with several servers, change the key in the key.bin as follows:

1. Go to the writer machine in the JMX. You can choose any machine in the cluster and click on the writer link on the top of each page.
2. In the UCMDB section of the console, click UCMDB:service=Discovery Manager.

3. Change the key in one of the following ways:

   • Click changeEncryptionKey (this imports the existing encryption key)

   • Click generateEncryptionKey (this generates a random encryption key)

4. On the writer machine, go to the file system and find the key.bin at: C:\UCMDB\UCMDBServer\conf\discovery\key.bin
5. Copy the key.bin from the location on the writer machine to each one of other machines in the cluster to the folder: C:\UCMDB\UCMDBServer\conf\discovery\customer_1\ and rename the destination file (for example, key_new.bin).

6. For each of the other servers (readers) do the following:

   1. Switch the reader to be a writer (you can do this from the High Availability JMX) and wait until it changes.

   2. Connect to the JMX of the current writer and click UCMDB:service=Discovery Manager.

   3. Click and invoke changeEncryptionKey, use the same details you entered in step 3 (for newKeyFileName, use the new name you assigned at step 5).

   4. Verify that you get the following message: Key was created successfully.

High Availability Cluster Message Encryption

Use cluster message encryption to encrypt all the messages in the cluster.

To enable cluster message encryption:

1. In UCMDB, go to Administration > Infrastructure Settings Manager.

2. Find the setting Enable High Availability cluster communication encryption and set it to true.

3. Provide a secret key for symmetric encryption on all the servers. The key should be placed in a keystore of type JCEKS in the following location <UCMDB installation folder>\conf\security\cluster.encryption.keystore.

   Note The UCMDB comes with this keystore pre-configured out of the box. This keystore is the same for all clean UCMDB installations, and thus not secure. If you wish to securely encrypt cluster messages, please delete this file, and create a new one by following this procedure.

4. From <UCMDB installation folder>\bin\jre\bin, run the following command:

   Keytool –genseckey –alias hpcert –keystore <UCMDB installation folder>\conf\security\cluster.encryption.keystore –storetype JCEKS

5. You will be asked for the new keystore password. The default password is “hppass”. If you want to use a different password, you need to update the server by running the following JMX method:

   UCMDB:service=High Availability Services: changeClusterEncryptionKeystorePassword

6. Replace the old <UCMDB installation folder>\conf\security\cluster.encryption.keystore of all the servers in the cluster with this new keystore.

7. Restart the servers.

Troubleshooting - High Availability Mode

Upon every startup of the UCMDB server, the server sends a test message to the cluster to verify if it successfully connected to the cluster. If there is a problem with the connection, the message fails and the server is stopped to avoid the whole cluster getting stuck.

Some examples of wrong cluster encryption configuration are:

• Disabled encryption on one node when another node enabled it.
• Wrong or missing cluster.encryption.keystore
• Wrong or missing key in the keystore

If the server gets stuck because of a configuration issue, the error message is:

2012-09-11 17:48:23,584 [Thread-14] FATAL - #### Server failed to connect properly to the cluster and its service is stopped! Please fix the problem and start it again ####

2012-09-11 17:48:23,586 [Thread-14] FATAL -       Potential problems can be: wrong security configuration (wrong or missing cluster.encryption.keystore, wrong key, disabled encryption in a cluster with enabled encryption)