Data Flow Probe Auto Upgrade

• Limitation: If the auto upgrade fails, retry will not resolve the issue. You need to access the corresponding probe server and perform manual deployment of version 10.33 probe.
• Known Issue: The C:\hp\UCMDB\temp folder was created and used by the probe auto upgrade agent during the upgrade process. If you see this folder on your probe server, you can just ignore it, or safely remove it. It has no functional impact.

• Check if resources are placed under the right place after UCMDB server is upgraded to version 10.33

  1. Check if the Data Flow Probe installer is placed under the right place

     Go to the <UCMDB_Server>\content\probe_installer directory. This directory should contain the probe installer UCMDB_DataFlowProbe_10.33.exe.

  2. Check if the probe auto upgrade agent package is placed under the right place

     Go to the <UCMDB_Server>\runtime\probe_upgrade directory. This directory should contain the probe upgrade package probe-patch-windows.zip.

     If the probe-patch-windows.zip package does not exist,

     1. Go to <UCMDB_Server>\content\probe_patch.
     2. Copy the probe-patch-10.33-windows.zip package to the <UCMDB_Server>\runtime\probe_upgrade directory.
     3. Restart the UCMDB server. UCMDB server will then perform probe auto upgrade.

• Probe auto upgrade log files

  Check the probe auto upgrade log files (in the <DataFlowProbe>\runtime\log directory) for more details:

  • pg_upgrade.log. Shows the running details of the pg_upgrade.bat script, including the details about PostgreSQL upgrade and table splitting.
  • probe_upgrade_conf_merge.log. Shows the related information when probe installer merger configuration files.
  • probe_auto_upgrade.log. In the probeUpgradeLogs subfolder, shows the related information when the probe auto upgrade agent upgrades a probe.

  For more details about the log files, see Data Flow Probe Log Files.

• XML Enricher service port conflict issue

  Problem: The XML Enricher service may fail to start after the probe upgrade due to port conflict. In that case, the probe_auto_upgrade.log is placed under the failed folder, for example, <UCMDB_Server>\runtime\log\probeUpgradeLogs\10.22to10.33\failed. You can find the following message in probe_auto_upgrade.log:

  2017-07-14 11:27:11 INFO  ServiceControl:106 - Starting XML Enricher service...

  2017-07-14 11:27:11 INFO  ServiceControl:328 - XML EnricherStatus status: STOPPED

  2017-07-14 11:27:11 INFO  ServiceControl:381 - Waiting for execution...

  2017-07-14 11:27:46 ERROR ServiceControl:394 - Problems occurred during execution.

  Solution: Check <DataFlowProbe>\runtime\logWrapperEnricher.log, if you find "Port already in use: 34545", you can change the port for XMLEnricher by editing the <DataFlowProbe>\bin\xmlenriche\WrapperEnricher.conf file, or free the port 34545.

• When PostgreSQL finishes upgrade, you can check the PostgreSQL version to verify if the upgrade is successful or not.

  In a more general way, you can check the pg_upgrade.log in the <DataFlowProbe>\runtime\log folder for more details.

  If PostgreSQL upgrade is completed successfully, you can find “The new PostgreSQL will be used” message in the pg_upgrade.log file, and you can also see two folders: <DataFlowProbe>\pgsql and <DataFlowProbe>\pgsql.old . The <DataFlowProbe>\pgsql.new folder was removed when the upgrade is completed successfully. If you manually run the script from the <DataFlowProbe>/tools/dbscripts folder to upgrade the database again, the log will tell you that pgsql.new does not exist, and running the script again has no functional impact to the PostgreSQL installation.

• In some cases the PostgreSQL upgrade may fail. Then you can find three subfolders under <DataFlowProbe>: pgsql, pgsql.old, and pgsql.new. You can also find more details in the pg_upgrade.log file, which displays messages that may indicate why the upgrade failed. You may follow the solutions for different log messages.

  1. Log message: Folder pgsql.new doesn't exist.

     • Possible Cause: Something unexpected happened when installing the probe, and the probe failed to generate the pgsql.new folder.

       Solution: Download PostgreSQL resources for the same version from the official PostgreSQL website and extract the resources to the pgsql.new folder, then rerun the pg_upgrade.bat script.

     • Possible Cause: You have already run the script more than once, and the script already deleted the pgsql.new folder previously.

       Solution: The PostgreSQL upgrade is completed successfully previously. Just check for the PostgreSQL version.

  2. Log message: The new PostgreSQL database initialization failed.

     Possible Cause: The conditions for initdb were not met.

     Solution: Check if the password is correct, or there is no data folder in pgsql.new.

  3. Log message: The precheck of the old and new PostgreSQL failed.

     Possible Cause: The script did not run in the local system account or has no full control of the files.

     Solution: Switch to the local system account, or add full control to the whole folder for users, then rerun the script.

  4. Log message: PostgreSQL upgrade failed, the old PostgreSQL will still be used.

     Possible Cause: The conditions for pg_upgrade.exe were not met.

     Solution: Check the conditions for both the old PostgreSQL and the new PostgreSQL, make sure both are fine. You can manually run the following command to find more details:

     "%DB_PATH%\pg_upgrade.exe" -b "%BASE_DIR%\pgsql\bin" -B  "%BASE_DIR%\pgsql.new\bin" -d "%BASE_DIR%\pgsql\data" -D "%BASE_DIR%\pgsql.new\data" -p 5436 -P 5437 -U postgres

  5. Log message: Table splitting failed, the old PostgreSQL will still be used.

     Possible Cause: There is no ddm_discovery_results table in the database, or the upgrade failed when creating the ddm_discovery_touch_results table.

     Solution: Check the log details to find out where the problems happened, then check the script tools\dbscripts\migrateData.cmd.

  After resolving issue a ~ issue e above, you can follow the steps below to upgrade PostgreSQL manually:

  1. Stop the UCMDB_Probe_DB service.
  2. Remove the content of the pgsql folder and copy the content of the pgsql.old folder into the pgsql folder.

  3. Give full control to the user of the DataFlowProbe folder, and then from the <DataFlowProbe>/tools/dbscripts folder run the following command:

     pg_upgrade.bat %DB_Password%

  4. Once the command is successful run, revert the full control you granted to the user.

Before upgrading a probe, the probe auto upgrade agent checks the environment and the probe status.

a. Check probe version. Only a probe of version 10.22 or higher could be upgrade. For supported probe versions, see Supported Versions.

b. Check probe status. Only union mode probe in non-FIPS mode could be upgraded.

c. Check available disk space. At least 10 GB disk space is required to perform the probe auto upgrade.

If a probe does not satisfy the requirements, it will be restored and back on running.

• How to check if a probe has been upgraded successfully?

  You can check the probe_auto_upgrade.log file (in the C:\hp\UCMDB\DataFlowProbe\runtime\log\probeUpgradeLogs folder).

  • If the probe has been upgraded successfully, you can see the following message in the log file:

    Finished probe upgrade. Probe has been upgraded to [version] [Build]. Probe auto upgrade agent will exit.

  • If the probe upgrade failed, there is no upgrade related error message in the probe_auto_upgrade.log file.

    For further information about the upgrade failure, check the following log files:

    • probe_upgrade_conf_merge.log
    • pg_upgrade.log
    • probe_post_upgrade.log

    For information about the probe log files, see Data Flow Probe Log Files.

  • Problem: Sometimes due to the environment, the probe installer may be in hung state and cannot finish the upgrade. If this happens, the probe auto upgrade agent will abort the probe upgrade process and restore the probe.

    Solution: You need to manually upgrade the probe.

  • Problem: Log shows that “errors occurred installing probe”, and probe service, probe DB service, or XML Enricher service could not be started. It may happen when errors occur launching the probe installer.

    Solution: You need to manually upgrade the probe.

    Most likely it is caused by the missing of some properties in the configuration file. If not, you may need to check the following log files for further information:

    • probe_upgrade_conf_merge.log
    • pg_upgrade.log
    • probe_post_upgrade.log

When the probe installer is launched, it will merge the following configuration files:

• DataFlowProbe.properties
• DataFlowProbeOverride.properties (If exists)

The result is that all the custom configuration settings will be written into the DataFlowProbeOverride.properties file.