Reviewing Later Versions of Software

Part of the benefit of reviewing your software with the Review Tool is the leverage it provides for reviews of subsequent versions of your software, or reviews of similar software.

Creating a Review for Later / Different Software

In reusing a review, it’s best to start by copying the existing review so that the original is preserved. To create the new copy of the review:

1) Create a new review. This could be an empty review, containing no checks, but it needs to be based on the same checklist as the existing review.

2) Import the current review into the new review. Use the Review > Update [New_Review] > Import Review into [New_Review]... dialog, select the current review, and choose Import Full Review.

3) Recompute the review. From the Review display, invoke Clear All Automatically Generated Steps and then Run Checks Automatically in order to ensure that the changes in the software are taken into account in the calculations.

At this point, you have the option to import the probe ratings from the original review. You may decide to re-assess all of the probes in the new review. But importing probe ratings enables you to focus your review effort on assessing just those probes that are new or have changed between the reviews. Probe ratings can be imported from the original review through the Review > Update [New_Review] > Import Probe Ratings into [New_Review]... dialog

Importing and Exporting Probes Files

Standard probe files offer a way of automating the manual identification of probes. These files contain lists of symbols, typically functions. Lines in source files, definitions of critical regions, and notes (as probes) can also be specified in the files. Each item in a standard probe file acts as a candidate for a probe in the corresponding check / step. Automation in the Review Tool can process these files and identify any project symbols, source file lines, etc, that match the candidates in the file. When a match is found, an appropriate probe is generated.

Two standard probe files can exist for a given check / step that requires user input. One file can exist at the checklist level. This is normally the more generic of the two files, containing symbols that are common to C/C++ libraries and that are often an appropriate candidate probe for the check/step being reviewed. Any standard probe files distributed as part of the Review Tool distribution are included at the checklist level. Because they are generic, these standard probes files can sometimes list more probes than are appropriate for the current project. Therefore the results of importing the probe file into the step should be examined and any unnecessary probes should be removed. Checklist level files are located in the directory ../imagix/user/checklists/[checklist_name].std.

A second standard file can exist at the project level. This probe file is normally the more targeted of the two, containing probe candidates that are specific to the project. It is in these project level files that lines of source code and definitions of critical regions are more likely to be specified. Project level files are located in the directory ../[your_project].4D/reviews/[checklist_name].std.

When standard probe files are automatically loaded, and they exist at both the checklist and project levels, the one at the project level takes precedence.

Additional probe files, not specific to a particular check / step, can be imported through more of a manual process. Within a Check display, the Modify > Import Probes from Imagix Probe File... dialog enables you to choose any file and import it to generate probes for the current step.

A similar process can be used to import specific results from complementary static analysis tools. Through the Modify > Import Probes from SARIF File... dialog, you can use SARIF to load defects identified by an external static analyzer, and apply them as probes in your current check.

The Help button on the dialog documents the format of the (standard) probe files. Following this format, you're able to create and modify any probe files. You can also export probes to generate probe files that represent the current probes for a step. Such files are placed with the project level standard files. Exporting can be a useful way to leverage any work that you've put into manually identifying probes for a particular step. The exported probe file can be loaded if you rerun that step, either in the current review or in a related review. Any of the probe files you create, modify or export can be placed in the checklist or projects locations, to enable more automatic loading.

Comparing Results

The Review Tool automates many of the steps in generating probes. The use of probe files can automate further steps. The final step in a check, assessing the resulting probes, remains somewhat effort intensive.

The ability to compare results from different reviews can reduce some of this work. If the software has been reviewed previously, and you're now reviewing a later or related version of the software, you can compare the results to find what has changed. You can focus your assessment efforts on those probes which are unique to the current review.