Administer > System Security > FIPS mode > Configure legacy FIPS mode in Service Manager

Configure legacy FIPS mode in Service Manager

User Roles: System Administrator

Note Prior to version 9.50, Service Manager supported only FIPS 140-2 compliant data encryption. For backward compatibility, the legacy FIPS mode configuration described in this topic is still supported. The information in this topic applies only when the legacy fipsmode:1 configuration is used. If your system has not yet FIPS mode enabled, HPE strongly recommends you to ignore this topic and see FIPS mode instead for details about the enhanced support of FIPS mode, which was introduced in Service Manager 9.50.

To run Service Manager in FIPS mode, you need to configure both the server and clients. By default, FIPS mode is disabled, and Service Manager uses the 64-bit DES data encryption algorithm. This section describes how to enable FIPS 140-2 compliant data encryption in the SM server, Windows Client and web clients, by using fipsmode:1 in the Server\RUN\sm.ini file.

Note To enable FIPS mode in a scaling environment, you must configure FIPS mode in each of the SM server nodes and clients (Windows, web, SRC, and Mobility).

Server side

To enable FIPS mode in the server:

  1. Make sure your server and applications have upgraded to version 9.32 or later.
  2. Make a backup of your database.

    Caution FIPS mode requires the database to upgrade to the 256-bit AES encryption algorithm. Because the upgrade is irreversible, make a backup of your database.

  3. Check the length of each encrypted field in your database, and increase the length if needed.

    Tip Use the following formula to determine a safe length for an AES encrypted field (in bytes):

    Safe_Length=32 + 2 * source_string_length

    Where: source_string_length is the length of the source string (for example, if you enter an 8-character password, the source string length is 8 bytes).

  4. Upgrade your applications to the AES data encryption algorithm.

    FIPS mode requires the use of the 256-bit AES data encryption algorithm for the database. By default, the Service Manager applications uses the 64-bit Data Encryption Standard (DES) encryption algorithm. You must upgrade the database from the DES encryption algorithm to AES by running the sm -upgradeencralg command.

    Caution This algorithm upgrade is required only when you want to enable FIPS mode.

    1. Make sure you can connect to the server successfully.
    2. Make sure all Service Manager server processes are stopped.
    3. Check that the server's configuration file (sm.ini) does not include the fipsmode parameter.
    4. Run the following command:

      sm -upgradeencralg

      Note If you have many tables with encrypted fields, running this command may take quite a while; if you have already run this command on the database, running it again will have no effect and do no harm to the database.

      Tip You can specify a 32-character alphanumeric value for this parameter, as the new encryption key; if you do not specify any value, a default 32-character (256-bit) encryption key is used. You can also change your encryption key later using the changeencrkey parameter.

    5. Check the server log (sm.log) for any errors.
  5. In the server configuration file (sm.ini), set the following parameter.

     fipsmode:1
  6. Start the server.

    Note If your server fails to start, and a message occurs in the server log that indicates the system failed to load libeay32.dll, rename the libeay32_rba.dll (in the RUN folder) to libeay32.dll and overwrite the existing libeay32.dll file in the same folder, and then restart the server.

Client side

You need to plug in a FIPS-certified third-party JCE provider to the Service Manager web tier and Windows client, as described in the following. Out-of-the-box, the RSA BSAFE JCE provider is provided but disabled. You can enable the RSA BSAFE JCE provider or configure another JCE provider if you like.

Before you proceed, make sure you have the following information or files available. For information about any JCE providers other than RSA BSAFE, refer to the specific JCE provider documentation.

Item Value or File(s)
JCE provider name

For RSA BSAFE:

JsafeJCE

JCE provider class name

For RSA BSAFE:

com.rsa.jsafe.provider.JsafeJCE

JCE provider's jar file(s) required for FIPS mode

For RSA BSAFE, the following files are already bundled in both the SM web tier and Windows client:

cryptojcommon.jar
cryptojce.jar 
jcmFIPS.jar
JCE Unlimited Strength Jurisdiction Policy Files

To run in FIPS mode, both of the Windows and web clients need to update to the JCE Unlimited Strength Jurisdiction Policy Files (local_policy.jar and US_export_policy.jar), which you can download from Oracle:

  • http://www.oracle.com/technetwork/java/javase/downloads/jce-7-download-432124.html (for JRE 7) 
  • http://www.oracle.com/technetwork/java/javase/downloads/jce-6-download-429243.html (for JRE 6)

Caution The JCE Unlimited Strength Jurisdiction Policy Files are subject to import/export restrictions. HPE recommends you consult the export/import control counsel in your country to determine if you are allowed to use these files.

Note The following steps use the RSA BSAFE JCE provider as an example; if you want to use another JCE provider, refer to the specific JCE provider documentation for information about the correct parameter values and jar file(s).

To configure FIPS mode in the web client:

  1. Open the web tier configuration file (web.xml) in a text editor.
  2. Locate the following lines:

    <init-param>
        <param-name>JCEProviderName</param-name>
        <param-value/>
    </init-param>
    <!-- Specify the JCE Provider class name here, the full name includes the package name -->
    <init-param>
        <param-name>JCEProviderClassName</param-name>
        <param-value/>
    </init-param>
  3. Specify the JCE provider name and JCE provider class name.

    The following are the names for RSA BSAFE.

    <init-param>
        <param-name>JCEProviderName</param-name>
        <param-value>JsafeJCE</param-value>
    </init-param>
    <!-- Specify the JCE Provider class name here; the full name includes the package name -->
    <init-param>
        <param-name>JCEProviderClassName</param-name>
        <param-value>com.rsa.jsafe.provider.JsafeJCE</param-value>
    </init-param>

    Caution Both parameter values are case-sensitive.

  4. Copy the required jar file(s) to the web tier's WEB-INF/lib folder:

    Note This step is needed only for a JCE provider other than RSA BSAFE. For RSA BSAFE, the required jar files (cryptojcommon.jar, cryptojce.jar, and jcmFIPS.jar) are already in the folder.

  5. Copy the JCE Unlimited Strength Jurisdiction Policy Files you downloaded to the web tier server's jre\lib\security directory (for example: C:\Program Files (x86)\Java\jre7\lib\security) to overwrite the two existing jar files.
  6. (For web application servers running on Unix only) Use one of the following solutions to avoid potential performance issues.

    Solution Description
    1

    Add a parameter -Djava.security.egd=file:/dev/./urandom to JAVA_OPTS of the web application server.

    For example, add the following line in Tomcat's catalina.sh file:

    JAVA_OPTS="$JAVA_OPTS -Djava.security.egd=file:/dev/./urandom"
    2

    Modify the <JRE_HOME>/lib/java.security file used by your web application server to set securerandom.source as the following:

    securerandom.source=file:/dev/./urandom
  7. Restart your web application server for your configuration to take effect.

  8. In your web tier's sm.log file, search for the following message (which indicates you have successfully configured FIPS mode):

    The specified JCE provider <jceProviderName> has been initialized successfully.

    Note If you failed to configure FIPS mode, the following message occurs instead:

    Initialization of the specified JCE provider <jceProviderName> failed.

To configure FIPS mode in the Windows client:

Note For RSA BSAFE, skip steps 1 and 2. This is because the required modifications are already implemented for RSA BSAFE out-of-the-box.

  1. Copy the JCE provider's required jar file(s) to the Windows client's third-party plugins folder: <Service Manager installation path>\Client\plugins\com.hp.ov.sm.client.thirdparty_9.32.xxxx\.

  2. Update the Manifest.mf file in the following folder: <Service Manager installation path>\Client\plugins\com.hp.ov.sm.client.thirdparty_9.32.0002\META-INF.

    1. Open the Manifest.mf file in a text editor.
    2. Add the required jar file name(s) to the Bundle-Classpath section.

      Out-of-the-box, the jar files (cryptojcommon.jar, cryptojce.jar, and jcmFIPS.jar) for RSA BSAFE are already included as shown below. If you want to use another JCE provider, add its required jar file names after the BSAFE ones.

      Bundle-ClassPath: lib/,
       ...
       saaj-impl-2.1.jar, 
       FastInfoset-jwsdp-2.0.jar,
       cryptojcommon.jar,
       cryptojce.jar,
       jcmFIPS.jar,
       <another JCE provider jar file name>,
       ...

      Caution Be sure to include one leading whitespace before each jar file name.

    3. Add the JCE provider's package name to the Export-Package section.

      For example, for RSA BSAFE the package name com.rsa.jsafe.provider is already included as shown below:

      Export-Package: .,
       COM.rsa.Intel,
       COM.rsa.asn1,
       COM.rsa.jsafe,
         com.rsa.jsafe.provider,
       ...
  3. Configure the Windows client preferences.

    1. In the Windows client, click Window > Preferences > HPE Service Manager > Security.
    2. Specify these parameter values (the following are values for RSA BSAFE):
      • JCE provider name: JsafeJCE
      • JCE provider class name: com.rsa.jsafe.provider.JsafeJCE
  4. Copy the JCE Unlimited Strength Jurisdiction Policy Files you downloaded to the <Service Manager installation path\Client\jre\lib\security directory to overwrite the two existing jar files.
  5. Make a backup of your Windows client configuration cache, which is located in the Home folder (for example, C:\Users\<username>\ServiceManager), and then clear the cache.

  6. Restart the Windows client for your configuration to take effect.
  7. In your Windows client's sm.log file, search for the following message:

    The specified JCE provider <jceProviderName> has been initialized successfully.

    Note If you failed to configure FIPS mode, the following message occurs instead:

    Initialization of the specified JCE provider <jceProviderName> failed.