Configure Java for FIPS mode

To enable Service Manager (SM) to run in FIPS mode, you must configure the JRE instances that the SM components use. The following describes the JRE configurations required for each SM component.

Server

For the Service Manager Server, you only need to install Oracle JRE 8 and update the Oracle JRE to use the unlimited strength JCE policy files appropriate for your specific JRE instance. For detailed instructions, see JRE support and Download JCE unlimited strength policy files.

Windows Client

The Windows Client comes with an embedded JRE (OpenJDK 8). You need to do the following:

• Replace the OpenJDK JRE with Oracle JRE. For details, see JRE support.
• Update the two policy files in the <Service Manager installation path>\Client\jre\lib\security directory with the unlimited strength policy files you have downloaded from Oracle. For detailed instructions, see Download JCE unlimited strength policy files.

Web Tier

The configuration steps vary with the web application server on which the Service Manager Web Tier is deployed.

For Tomcat

If the SM Web Tier is deployed on any of these web application servers, perform the following JRE configuration steps:

1. Copy the CryptoJ jars (cryptojce-x.x.x.jar, cryptojcommon-x.x.x.jar and jcmFIPS-x.x.x.jar) from the Web Tier's \WEB-INF\lib directory to your \jre\lib\ext directory.

   Note In the jar file names, the placeholder ("x.xx") represents the version number of the jar files, which may vary with each release.

   For example:

   From: C:\Program Files\Apache Software Foundation\Tomcat 7.0\webapps\webtier-9.xx\WEB-INF\lib

   To: C:\Program Files\Java\jre1.8.0_65\lib\ext or C:\Program Files\Java\jdk1.8.0_65\jre\lib\ext

2. Modify the jre/lib/security/java.security file as described in the following steps:

   1. Configure the JsafeJCE provider by updating and rearranging the list of the providers as shown in the following (where changes are highlighted in bold).

      security.provider.1=com.rsa.jsafe.provider.JsafeJCE

      security.provider.2=sun.security.provider.Sun

      security.provider.3=sun.security.rsa.SunRsaSign

      security.provider.4=sun.security.ec.SunEC

      security.provider.5=com.sun.net.ssl.internal.ssl.Provider JsafeJCE

      security.provider.6=com.sun.crypto.provider.SunJCE

      security.provider.7=sun.security.jgss.SunProvider

      security.provider.8=com.sun.security.sasl.Provider

      security.provider.9=org.jcp.xml.dsig.internal.dom.XMLDSigRI

      security.provider.10=sun.security.smartcardio.SunPCSC

      security.provider.11=sun.security.mscapi.SunMSCAPI

   2. Configure Crypto-J for FIPS 140-2 compliant operations by adding the following lines (for example, you can add them to the end of the file):

      com.rsa.cryptoj.fips140initialmode=FIPS140_SSL_MODE
      com.rsa.cryptoj.kat.strategy=on.load

   3. Configure a FIPS validated random number generation algorithm as the default one for Crypto-J, by adding the following line (for example, you can add it to the end of the file):

      com.rsa.crypto.default.random=HMACDRBG256

      Note Crypto-J uses HMACDRBG256 as the default random algorithm when no other random number generation algorithm is specified. If required, you can change this value to another valid value, for example, HMACDRBG192.

   4. Update the keystore.type property to PKCS12 as follows:

      keystore.type=PKCS12

3. Replace the policy files in the jre\lib\security folder with the JCE Unlimited Strength Jurisdiction Policy Files. For detailed instructions, see Download JCE unlimited strength policy files.

For WebSphere Application Server

Task 1: Configure WAS JRE for FIPS mode

If the SM Web Tier is deployed on WebSphere Application Server, perform the following JRE configuration steps:

1. Configure the FIPS provider in the java.security file.

   WebSphere Application Server provides a FIPS-validated provider named IBMJCEFIPS. Use this provider when enabling FIPS mode in WebSphere Application Server. To do this, edit the java.security file in [WAS_HOME]\ java\jre\lib\security to include IBMJCEFIPS as the FIPS provider. In this example, we are considering the use of IBM SoftwareDevelopment Kit (SDK).

   1. Include the FIPS provider in the list of providers:

      security.provider.1=com.ibm.crypto.fips.provider.IBMJCEFIPS

   2. Rearrange the rest of the providers in the list as shown in the following:

      #
      # List of providers and their preference orders (see above):
      #
      security.provider.1=com.ibm.crypto.fips.provider.IBMJCEFIPS 
      security.provider.2=com.ibm.crypto.pkcs11impl.provider.IBMPKCS11Impl 
      security.provider.3=com.ibm.crypto.provider.IBMJCE 
      security.provider.4=com.ibm.jsse2.IBMJSSEProvider2 
      security.provider.5=com.ibm.security.jgss.IBMJGSSProvider 
      security.provider.6=com.ibm.security.cert.IBMCertPath 
      security.provider.7=com.ibm.security.cmskeystore.CMSProvider 
      security.provider.8=com.ibm.security.jgss.mech.spnego.IBMSPNEGO 
      security.provider.9=com.ibm.security.sasl.IBMSASL 
      security.provider.10=com.ibm.xml.crypto.IBMXMLCryptoProvider 
      security.provider.11=com.ibm.xml.enc.IBMXMLEncProvider 
      security.provider.12=org.apache.harmony.security.provider.PolicyProvider

2. Replace the policy files in the jre\lib\security folder with the JCE Unlimited Strength Jurisdiction Policy Files. For detailed instructions, see Download JCE unlimited strength policy files.

Task 2: Configure FIPS mode in WAS

To run WebSphere Application Server (WAS) in FIPS mode, you must continue to perform the following additional configuration steps in WAS:

1. Enable FIPS through the WebSphere Application Server administrative console.
   1. Go to Security > SSL Certificate and Key Management > Manage FIPS.
   2. Select Enable FIPS 140-2, and click Apply.
   3. In the next view, click Save.
2. Configure FIPS properties in the ssl.client.props file. To do this, edit the ssl.client.props file in the [WAS_HOME]\ profiles\AppSrv01\properties folder. In this example, we use AppSrv01 as the WebSphere profile.

   1. Change the com.ibm.security.useFIPS property from false to true:

      com.ibm.security.useFIPS=true

   2. Ensure that the com.ibm.ssl.protocol property is set to TLS. To do this, change the com.ibm.ssl.protocol property from SSL_TLS to TLS:

      com.ibm.ssl.protocol=TLS

3. Restart WebSphere Application Server.

Service Request Catalog (SRC)

SRC must be deployed on Tomcat. You need to configure the JRE used by SRC's Tomcat for FIPS mode. The steps are the same as those for the SM Web Tier. For details, see the steps for Tomcat in the Web Tier section.

Mobility Client

The Mobility Client can be deployed on either Tomcat or WebSphere. You need to configure the JRE used by the web application server (Tomcat or WebSphere) for FIPS mode. The steps are the same as those for the SM Web Tier. For details, see the steps for Tomcat or WebSphere in the Web Tier section.

Solr Search Engine

The Solr Search Engine comes with an embedded Tomcat instance. To enable FIPS mode, you need to configure the JRE used by the embedded Tomcat instance. The steps are the same as those for the Web Tier. For details, see the Tomcat part in Web Tier.

Openfire Chat Server

To run Service Manager Collaboration in FIPS mode, you must update the 32-bit Oracle JRE 8 that you installed for the Openfire chat server.

To do this, you need to copy the JCE Unlimited Strength Jurisdiction Policy Files to the jre/lib/security/ directory to overwrite the two existing policy files. For detailed instructions, see Download JCE unlimited strength policy files.

Follow these JRE configuration steps:

1. Copy the CryptoJ jars (cryptojce-6.2.jar, cryptojcommon-6.2.jar and jcmFIPS-6.2.jar) from the Web Tier's \WEB-INF\lib directory to your \jre\lib\ext directory. For example:

   From: C:\Program Files\Apache Software Foundation\Tomcat 7.0\webapps\webtier-9.xx\WEB-INF\lib

   To: C:\Program Files\Java\jre1.8.0_65\lib\ext or C:\Program Files\Java\jdk1.8.0_65\jre\lib\ext

2. Modify the jre/lib/security/java.security file as described in the following steps:

   1. Configure the JsafeJCE provider by updating and rearranging the list of the providers as shown in the following (where changes are highlighted in bold).

      security.provider.1=com.rsa.jsafe.provider.JsafeJCE

      security.provider.2=sun.security.provider.Sun

      security.provider.3=sun.security.rsa.SunRsaSign

      security.provider.4=sun.security.ec.SunEC

      security.provider.5=com.sun.net.ssl.internal.ssl.Provider JsafeJCE

      security.provider.6=com.sun.crypto.provider.SunJCE

      security.provider.7=sun.security.jgss.SunProvider

      security.provider.8=com.sun.security.sasl.Provider

      security.provider.9=org.jcp.xml.dsig.internal.dom.XMLDSigRI

      security.provider.10=sun.security.smartcardio.SunPCSC

      security.provider.11=sun.security.mscapi.SunMSCAPI

   2. Configure Crypto-J for FIPS 140-2 compliant operations by adding the following lines (for example, you can add them to the end of the file):

      com.rsa.cryptoj.fips140initialmode=FIPS140_SSL_MODE
      com.rsa.cryptoj.kat.strategy=on.load

   3. Configure a FIPS validated random number generation algorithm as the default one for Crypto-J, by adding the following line (for example, you can add it to the end of the file):

      com.rsa.crypto.default.random=HMACDRBG256

      Note Crypto-J uses HMACDRBG256 as the default random algorithm when no other random number generation algorithm is specified. If required, you can change this value to another valid value, for example, HMACDRBG192.

   4. Update the keystore.type property to PKCS12 as follows:

      keystore.type=PKCS12

3. Replace the policy files in the jre\lib\security folder with the JCE Unlimited Strength Jurisdiction Policy Files. For detailed instructions, see Download JCE unlimited strength policy files.

IdM Service

The IdM service must be deployed on Tomcat. You need to configure the JRE used by the IdM Tomcat for FIPS mode. The steps are the same as those for the SM Web Tier. For details, see the steps for Tomcat in the Web Tier section.

Next step:

Generate FIPS validated certificates for the SM Server and other components