Teaching Using Scan Files (UNIX)

This section is for users who are not familiar with the UNIX OS environment and some of the issues associated with dealing with UNIX files.

These are guidelines for best practice rather than hard and fast rules. As you gain familiarity with the various data analysis tools supplied with Universal Discovery, you can develop new ways of clustering files to meet specific needs.

The mechanics of teaching UNIX applications to a User SAI are exactly the same as for Windows applications. For example, Publisher, Application, Application Type, Release, Version, etc. need to be identified and applied to one or more files.

However, the information gathering exercise required for UNIX teaching can be challenging, since the files themselves do not provide the same richness of information as in Windows.

Teaching software under the UNIX operating system relying only on scan files requires a slightly different approach than that used with Windows. Here are some of the points to consider:

• No version data is available from the files. This means there is no direct confirmation as to the source of the file.
• There is a limited amount of information available from the OS installed application list. This is assuming built-in UNIX software install tools (such as, SWTOOLS) have been used for installation. Many applications like Oracle can use their own installer and, hence, do not appear in the OS installed applications list.
• More communication with local knowledge specialists (site administrators or staff who are familiar with the on-site applications) may be needed than with Windows. Ideally, these people should have solid UNIX systems-level knowledge.
• Analysis Workbench is used in a different way from Windows data analysis. The main difference is that rather than concentrate on files and their properties, the emphasis is on clustering files from the directories into an application.
• There is no single version of UNIX. See Flavors of UNIX for the different UNIX platforms that are supported by Universal Discovery.

Here are some recommendations you can consider incorporating into your teaching project:

Use Separate User SAI Files for UNIX Entries

It is preferable to use a separate User SAI for UNIX entries. The reasons for this are:

• Clearly identifies UNIX specific applications.
• It is quite likely that UNIX machines will be processed separately from Windows based machines so this allows a separate set of load options to be set.

Use Separate User SAI Files for Different UNIX Flavors

If there are numerous UNIX flavors on site, then it may be preferable to have separate SAIs for each of them (for example, Solaris, HP-UX, AIX).

Scan a Base Machine that has Only UNIX On It

Since UNIX installations can be highly configurable, the ideal situation would be to scan a base machine that only has the operating system installed on it. This provides a base line from which to work. This should be considered even when the Operating System is included in the UNIX Master, since site specific patches may have been applied.

However, it may not always be possible to scan a machine that has only been installed with the Operating System and no other applications. If one is available then this should be used to teach as many Operating System related files (for example, patches) as possible. This should be dealt with first.

• Tag the target machine and the selected files, and teach these files as the base operating system build. This then removes the files from the unknown list.
• If a clean build is not available, then files normally found within the Operating System directories should be selected for teaching first.

From a file recognition perspective, the binary executable types for each UNIX platform are distinct and easily recognizable (e.g., HP/PA Coff for HP-UX, SPARC ELF for Solaris, AIX Coff for AIX, x86 ELF for Linux). Directory structures and file names are typically the same across different UNIX platforms for a given application.

An overview of the process of UNIX Teaching using Analysis Workbench follows:

1. From discussions with local specialists, identify a range of applications that are known to be on the scanned machines.
2. Since the number of applications is likely to be much smaller than for Windows, it is useful to track the clustering of files by machine selected and base directories included in the cluster.

This can be used in further discussions with local specialists to determine and confirm the application details.

3. Determine the number of unknown files to discover from the scanned files. To do this:
• Load a full set of scan files into Analysis Workbench with the Master UNIX SAI.
• Select the Status column, right-click the header, and choose the Chart option.

This will give an indication of the level of unknown files.

4. Select machines with the largest number of unknowns. To do this:
1. In a Machines Window, create a column from System Data > Application Recognition > Unrecognized Files
2. Sort on this column, and then Tag machines with the largest unrecognized files counts. Save these machines as a Tag list for further use.
5. Unload the scans.
6. Take a selection of sample machines from each flavor of UNIX in turn.
7. Create a user SAI for each flavor of UNIX
8. Select and load a target scan file – use entries in the previously saved Tag list

OS Files

1. Identify the OS version on the target machine:

Hardware > Operating System Data > UNIX type and Host OS

2. Tag the OS directories
3. Teach the unknown files found in these directories using the OS details

Installed Applications

View the OS installed applications list in the Hardware tab of the Viewer.

If there are any that can be identified then teach these.

Recognized Applications

When a UNIX Master SAI is available, use this to identify as many of the files as possible.

If there are some recognized applications do the following:

1. In Analysis Workbench, select an application, tag it, and then check the Files Window and Directories Window to see if the are any sub directories that contain Unknown or Checkver files.
2. If they do contain such files, then a reasonable assumption is that the Unknown and Checkver files are from the same application—possibly patches, for example.
3. Cluster the files and add them to the SAI using same details as the existing files.

Directories

Having worked on these applications, turn your attention to the directories.

1. Check environment paths for execution paths (for example, PATH). These are likely to be the directories where the main applications are located.
2. Select directories for file clustering.
3. Cluster files and add them to the SAI.

Selection Techniques

Identify OS Version on the Target Machine

Use one of the following methods to identify the OS version on the target machine:

1. In Viewer, use the selection Hardware > Operating System Data > Host OS Name and Host OS Version
2. In an Analysis Workbench Machine window, select the appropriate column to display. This is used to establish the version for the files in the main operating system directories.

Tag OS Directories – See OS Directories List

Select main OS directories and cluster files to an application.

Select known applications and select their directories and check for unknown or Checkver files mixed in the same directory as known files.

View the OS Installed Applications List

Some UNIX flavors contain directory paths of the OS Installed applications.

In the Viewer Hardware tab, go to Operating System Data/OS Installed Applications to see which applications are logged under the Operating System.

Check Environment Paths for Execution Paths

On the Hardware and Configuration tab in the Viewer, use the Operating System Data > Environment information to check specific paths.

These will point to directories containing executable applications.

Choosing Directories to Target for File Clustering

In Analysis Workbench, select directories for file clustering using the following criteria:

• Directories with recognizable descriptions in the path name
• Directories with the largest number of unknown files

How to Find Directories with a Specific Name

1. In an Analysis Workbench Directories Window, right-click the Name column heading, and select Tag > Contains.
2. Enter the name or partial name in the Search String box.
3. Select Anywhere for the String Position.

How to Determine Which Directories Have the Most Unknown Files

1. In Analysis Workbench, ensure that the Directories Window has the Tagged Files column displayed.
2. Go to a Files Window, and tag files with Status = Unknown or Checkver.
3. Go to a Directories Window, and sort on the Tagged Files Column.

The result may appear confusing, because the file counts do not always display in file count order. This is because the directory displays the total count of files but is placed in the order of the tagged files. To obtain a total count of the files, right-click the Tagged Files column heading, and choose Sum.

The screen does not refresh the tagged files count correctly if the file tags are removed.

File Details

If there are multiple versions of an application that have shared files in the same directory, choose the oldest version first and then progressively teach the newer versions in order. This is on the basis that the older files have carried forward into the newer versions.

Check that the main file selection is consistent in order to count licenses. This may require more than one Main file to be selected.

Look for consistency under the same date for the same version of the application

Group the files under the known application. There may be more than one version of the application sitting in the same directory. In this case it will be necessary to try and allocate the files to each version based on date.

Recommendations

• Never use a script file type as a Main file because these can appear in different operating environments or not used on a particular machine. This could lead to incorrect license counts.
• Load one UNIX operating system flavor at a time. This is to ensure that if an application exists for different environments, that they are taught for the specific operating system. This is because the same file name can have widely differing file sizes depending on the operating system. The binary executable file types will also vary from one UNIX platform to another. For example, HP/PA Coff files should only exist in HP-UX applications, not in Solaris versions.
• Version numbers may need to be approximated e.g. 4.3.3.79, 4.3.3.12 and 4.3.3.26 etc. may need to be consolidated as 4.3.3.x

1. Identify a machine with the sample applications on it.
2. Select the base directory.

To tag all sub directories off a base directory:

1. Select base directory in Directory Window.
2. Go to Details pane display path.
3. Click on path and use Ctrl C to copy all or part of the path string.
4. Go to Directories Window and right-click the “Name” column header.
5. Select Tag > Contains > search string and Ctrl V to copy path string.
6. Click OK. This will tag all the directories with the base path string.
3. Tag the directory and sub-directories.
4. Apply directory filter to Files window.
5. Tag All files displayed in filtered display.
6. Add to SAI.

Use web search facilities to identify applications and publishers. It may be sufficient to enter the name of a file and it can be found in an on-line document that relates to an application.

It may be necessary to become familiar with the various acronyms associated with a product in order to identify it.

You can find the installed applications for different operating systems in the Hardware tab of the Viewer.

The installed applications are listed under:

Operating System Data>OS Installed Applications

The amount of detail provided varies between UNIX flavors. These mainly refer to OS drivers etc. but can be useful in determining version numbers when teaching OS support files and libraries.

The following table lists all standard UNIX directories in System V Release 4 (SVR4).

Directory	Description
/	The Root directory, which is always present and is the parent of all the files in the file system.
/dev	Contains device files including terminals, printers and hard disks
/etc	Contains system administration and configuration files, the most important of which is /etc/passwd
/home	Directory containing the home directories for all the users
/opt	Root directory for file systems used by add-on applications
/sbin	Programs used when booting the system
/spool	Contains the directories for spooling files
/spool/lp	Spooling files for the printer
/spool/uucp	Files queued for UUCP
/spool/uucppublic	Files placed by UUCP
/stand	Standard programs and configuration files used when booting
/tmp	Contains temporary files created by UNIX
/usr	User accessible programs
/usr/bin	Executable programs and utilities
/usr/ucb	Contains the BSD compatibility package
/usr/games	UNIX games
/usr/lib	Libraries for programs and programming languages
/var	Contains files whose content vary as the system runs
/var/admin
/var/mail
/var/news
/var/opt
/var/tmp
/var/uucp

Directory	Description
/	The Root directory, which is always present and is the parent of all the files in the file system.
/dev	Contains device files including terminals, printers and hard disks
/etc	Contains system administration and configuration files, the most important of which is /etc/passwd
/home	Directory containing the home directories for all the users
/opt	Root directory for file systems used by add-on applications
/sbin	Programs used when booting the system
/spool	Contains the directories for spooling files
/spool/lp	Spooling files for the printer
/spool/uucp	Files queued for UUCP
/spool/uucppublic	Files placed by UUCP
/stand	Standard programs and configuration files used when booting
/tmp	Contains temporary files created by UNIX
/usr	User accessible programs
/usr/bin	Executable programs and utilities
/usr/ucb	Contains the BSD compatibility package
/usr/games	UNIX games
/usr/lib	Libraries for programs and programming languages
/var	Contains files whose content vary as the system runs
/var/admin
/var/mail
/var/news
/var/opt
/var/tmp
/var/uucp

Other UNIX systems may contain other key directories

Directory	Description
/kernel	Contains OS kernel and related files for Solaris.
/X11	Contains files for the X Window windowing system.

Directories such as /opt or /var may be used for customer installations in the same way that Program Files directory is used in Windows. However, some applications choose their own directory structure and local configurations of applications may override the default directory paths. Again, local knowledge is useful here.

Scanner Configuration for UNIX Scanning

For scanning, include signature to assist with uniquely identifying the first 8K of a file.

The file details are to save UNIX executables (binary and script). File extensions can be included but this may lead to spurious recognition.

Selected directories can be ignored provided they are known to be data directories only. This can be included in the scanner generator, otherwise create an .override.ini file that contains the directory exclusions.

Setting up XML Enricher to Include the UNIX User SAI for Recognition

Follow these steps:

1. Go to Server > Administration > System Configuration > Scan processing > SAIs
2. Make sure that unix.zsai as well as the User SAI used for UNIX teaching are in the SAI list and that they are enabled.

The key load options are set as follows:

Option	Description
Load	Either Load ALL files. This includes non executables. Or Load Executable and Archived only. This will reduce the number of files to load and needing recognition may not include script files.
Recognition	Recognition method - SAI Recognition Settings – Use level 3 heuristics. Either Apply recognition to ALL files. Or Executable and Archived only. This will reduce the load time. In either case the SAI libraries to use are as follows: master.zsai contains key information for Operating System etc. unix.zsai this contains a mixture of Solaris, AIX, HP-UX, and Linux applications.
Display	Applications Either Operational View. Or Business View. Business view only displays applications in the Applications window that have a Main file in the Files window, while the Operational view displays ALL application entries found in the SAI. These may be just be Associated files or Check ver. Recommendation Use Operational View for teaching to see all the entries. Use Business view when reporting to concentrate on Applications that are licensable.

Option	Description
Load	Either Load ALL files. This includes non executables. Or Load Executable and Archived only. This will reduce the number of files to load and needing recognition may not include script files.
Recognition	Recognition method - SAI Recognition Settings – Use level 3 heuristics. Either Apply recognition to ALL files. Or Executable and Archived only. This will reduce the load time. In either case the SAI libraries to use are as follows: master.zsai contains key information for Operating System etc. unix.zsai this contains a mixture of Solaris, AIX, HP-UX, and Linux applications.
Display	Applications Either Operational View. Or Business View. Business view only displays applications in the Applications window that have a Main file in the Files window, while the Operational view displays ALL application entries found in the SAI. These may be just be Associated files or Check ver. Recommendation Use Operational View for teaching to see all the entries. Use Business view when reporting to concentrate on Applications that are licensable.

AIX

Acronym for Advanced Interactive Executive. A version of UNIX from IBM that runs on its Power Systems, including System p and System i.

Although AIX is derived from System V Release 4, it is POSIX-compliant and meets the Trusted Computer Base (TCB) Level C2 security.

One of the major enhancements AIX offers is Visual System Management (VSM), a graphical interface into the older System Management Interface Tool (SMIT). VSM contains four main elements: Print Manager, Device Manager, Storage Manager and Users and Groups Manager.

HP-UX

A version of UNIX for HP computers. HP-UX includes BSD extensions, including the networking commands, the Korn shell, and a version of emacs. VUE (Visual User Environment) is HP’s GUI with individual workspaces for different tasks, drag-and-drop functions, a text editor, color icon editor, as well as other productivity tools.

HP-UX also includes SAM (System Administration Manager) for common administration tasks, such as adding new users, installing and configuring peripherals, managing processes and scheduling jobs. Diskless computers, either clients or servers, can boot from the server and can support locally mounted file systems, so that each client has access to its own data files as well as being able to share files with others.

Solaris

A version of UNIX from Sun Microsystems that runs on Intel/AMD based computers and Sun SPARC workstations and servers.

Solaris is based on UNIX System V Release 4, and includes networking support. The latest version features Java Desktop System based on GNOME that comes with a large set of applications, including StarOffice.

Linux

A UNIX-compatible operating system originally developed for Intel-based PCs. It now exists for many other platforms as well, including PowerPC, and Sun SPARC.

Strictly speaking, Linux is the name of the operating system kernel. Other parts of the operating system are part of the GNU Project. Linux is a complete UNIX clone and supports the X Window system.

Mac OS X

Mac OS X is a UNIX based operating system from Apple. The core of the system is a POSIX compliant operating system named Darwin that provides the standard UNIX command line interface. On top of it, higher levels of the Mac OS X system are built, including the Aqua user interface. Mac OS X is the most popular general-purpose UNIX-like operating system currently in use.

Operating System	File Type	Description
All	Script	List of commands in a text file to be executed by an interpreter. The term script and shell program are often used interchangeably.
Solaris (SPARC/x86)	ELF	Executable and Linking format is the binary output produced by gcc compilers. It replaced an earlier format to take advantage of shared libraries.
AIX	Coff	Common Object File Format. A revision to the format of executable and object files to provide support for dynamically linked libraries. The COFF definition was replaced by the Executable and Link Format(ELF) in SVR4
HP-UX (HPPA, ia64)	ELF	Executable and Linking format is the binary output produced by gcc compilers. It replaced an earlier format to take advantage of shared libraries.
HP-UX (HPPA)	Coff	Common Object File Format. A revision to the format of executable and object files to provide support for dynamically linked libraries. The COFF definition was replaced by the Executable and Link Format(ELF) in SVR4
Linux	x86 ELF,
x86-64 ELF	Executable and Linking format is the binary output produced by gcc compilers. It replaced an earlier format to take advantage of shared libraries.
Mac OS X	Mach-O	Mach-O is short for Mach Object file format, the executable file format used on Mac OS X for executables and shared libraries. It is a derivation of the a.out format that offers more extensibility and faster access to information.