Creating and Performing a Review
Creating a New ReviewA review, for Imagix 4D, is a repository of checks that implement the rules against which you want to assess your software, along with the results of what you find. In performing a review, you'll first be creating a new review consisting of checks.
All review related activities are started from the main Review menu.
Once you've invoked Reviews > Create a New Review, you'll be guided through the following steps.
1) Select a Checklist for the New ReviewYou'll be presented a list of the checklists installed in your environment. Choose the checklist that most reflects the review you'd like to conduct. You'll later (in step 4) have a chance to add other checks into your review.
2) Select ChecksAll of the checks defined in the checklist are available for your review. These may represent a broader scope than what you want to assess in your review. Narrow the review by selecting just those checks you want to conduct.
3) Consider Prerequisite ChecksOnce you've selected the checks to include in the review, a dialog may appear giving you a chance to select additional checks. These are the checks that determine probes used as inputs to steps in the checks that you have already selected.
Often, these additional checks are preparatory checks that have been designed to minimize the manual work necessary to conduct a review. These preparatory checks contain common steps that can be shared between checks; these steps are separated into preparatory checks so that the results of these checks can be shared as downstream inputs among multiple checks. Such preparatory checks are identified through their names, Prep-xxx.
The currently selected checks require certain prerequisite checks to be part of the review. If you encounter the dialog listing additional checks to select, your choices are to a) go back to the previous dialog and remove any selected checks that have prerequisites, b) select the prerequisite checks and add them into the review, or c) ignore that prerequisite checks for now and later add them (in step 4) from another review where they have already been populated.
4) Consider Importing An Existing ReviewOnce you have finished selecting the checks to include in your review, the dialog will guide you through naming the review, and then open it for modification.
At this point, before you begin performing your new review, it might be appropriate to import an existing review. Reasons for doing so include:
(a) populating the new review with checks and results (probes, ratings and notes) from a common review, focused on the prerequisite checks, that you or other reviewers have already completed.
(b) starting from a previous version of the review that will now be updated because of a software version change or a checklist change
(c) expanding the scope of the review, by importing a review based on an additional checklist
(d) populating the new review with checks and results (probes, ratings and notes) from an overlapping review where some parts of the review have already been completed
Importing an existing review is done through the Review > Update [Your_Review] > Import Review into [Your_Review]... menu. In the resulting dialog, you'll be able to choose between importing the entire review and updating just those checks that exist in your current review. For (a) and (c), the desired checks don't yet exist in your review, and you'll need to choose Import Full Review. For (d), where there are a number of reviews that contain overlapping checks and you want to import just the latest results from another review with those overlapping checks, you should select Update Existing Results.
Performing a ReviewThe process of conducting a review consists to identifying and then assessing the portions of your software that are relevant to each check. You're guided through this process by the steps in each check. These steps are an ordered set of actions that include instructions and often some level of automation. The outcome of each step is the generation or rating of probes. Such probes often serve as inputs to downstream steps.
Within a given check, the order in which steps must be undertaken is fixed. But there are few limitations on the order in which checks may be conducted, the exception being that the prerequisite checks should normally be completed first.
Some steps rely heavily on programmer knowledge and intelligence, leveraging the visualization of Imagix 4D to identify the relevant portions of the software and create the probes. Other steps are highly automated, where upstream probes serve as inputs into Imagix 4D's analysis functionality, and output probes are automatically generated.
Performing a review is an iterative process. You'll find yourself alternating between manual identification of probes, and the automated analysis using that data to generate downstream probes.
A) Manual IdentificationA check including its ordered steps is presented in the Check display. For steps that involve manual creation of probes, the display includes a description of what to look for in your software. For example, a step might include the guidance "Determine functions that execute commands or load libraries. A typical example for this is the function system (http://www.cplusplus.com/reference/cstdlib/system/)."
At this point, you're able to apply the normal visualization functionality of Imagix 4D to study your source code and find any symbols or source code lines that meet the criteria. The Check display includes features for creating probes identifying these symbols or source code lines.
From whatever Imagix 4D tool you are using to study your software - perhaps a Graph window to visualize symbols and their dependencies, a File Editor to browse through your source code, or a Data Flow to analyze the data flow for a variable - you're able to create a probe to record that portion of your software. Through the Modify menu in the Check display, you can enable the addition of a probe, and then actually add it via a right mouse button action in whichever tool you are using. This can be done in any step. Also, for the manual identification steps, there are buttons to facilitate importing as probes the results of analysis you may have done in a Graph window.
Sometimes, assistance is available for these manual steps, in the form of standard probe files that accompany a given checklist. These step-specific files contain lists of symbols, typically functions, that are commonly applied with respect to the given step's criteria. For example, the standard probe file for functions that execute commands of load libraries includes 'system', 'execl', 'execle', 'execlp', 'execv', 'execve' and 'dlopen'. Loading the standard probe files through the Review Tool interface creates probes identifying any of these functions that are used in your project. You can accept the resulting probes as is, or use them to guide your further analysis. See Importing and Exporting Probe Files
B) Automated AnalysisFor the automated steps, the automation can be invoked at a couple of levels. In an individual Check display, you can invoke the automatic action for the current step. When you do so, you're able to see the probes that get created.
From the overall Review display, you can invoke "Run Checks Automatically". This invokes all of automated steps whose input probes have been defined. Note that it is possible to have an empty set of probes as an input. What determines whether input probes have been defined is whether the output probes in the upstream step have been accepted. This acceptance is done automatically for automated steps invoked through "Run Checks Automatically".
The results of the automated steps are always available for review and override. So if you decide that the probes created through the step automation are incomplete, you can supplement them by manually adding probes that identify additional portions of your software.
And if some of the probes identify irrelevant portions of your software, you can remove the probes from the results of the step, so that the irrelevant probes are not used as input probes in downstream steps. This is especially important for the preparatory steps as they will impact many other steps. Any irrelevant probes they contain can cause extra review work.
C) Rating ProbesTypically, the last step of a check is to examine each of the probes that has been input into that final step, and to make an assessment about whether the indicated portion software is problematic. This assessment is recorded as a rating of No Issue, Concern or Violation.
The Review Tool contains a number of features to help with this assessment. The probes are presented in the Check display for the current check, and the ratings are assigned there. Details about each probe can be viewed in the Probe display. All of these displays are integrated with Imagix 4D's normal tools, so that the full force of Imagix 4D's visualization and analysis can be employed in assessing the compliance of a given probe.
Typically, you will find yourself skipping back and forth among (A), (B) and (C). The Review Tools guides you forward, and there are a few requirements with respect to order. For example, it's typically best to start with the Prep- checks, as you'll not be able to proceed on the downstream checks / steps where the Prep- checks are required until you've completed the upstream Prep- checks. The steps within a given check are ordered, and they require that the previous step in that check be completed. In particular, the assessment and assignment of probe ratings (C) is the last step in a check, and can't be completed until the previous steps are completed.
But within these general constraints, the Review Tool provides flexibility in what order you go about the review. At any given point, the Review display indicates the status of each check. After running an automated analysis (B), you're able to see which checks are ready for rating probes (C) and which checks are at an initial or intermediate step, waiting on manual identification of probes (A). You can choose to focus next on any of these. For example, you might decide to rate the probes as soon as a given check reaches the probe review step. Or you might instead delay all probe rating until all of the checks have been completed to that point.
Managing a ReviewAs you conduct a review, you can use the Review display to track progress. You're able to see the status of each of the checks - how many steps have been completed and how many probes have been identified in each step. For checks that have reached the final rate probes step, you're able to see how many assessments have been made, and what the ratings have been.
At any given time, one copy of the review can be opened in Modify mode, where changes can be made. Others users are able track the progress, view details or import the review into a separate review by opening the review in Read mode.
Within the review, project-specific data is collected and recorded about each check. During the review process, this underlying data grows. Backups of the review are created both automatically and manually, and can later be used to restore a previous point.
These backups also offer a more detailed way to track review progress. Through the Compare Reviews function, you're able to compare the current state of a review with a backup from earlier, and see more specifically the changes between the two points.
While the review is on-going, you may determine that you'd like to expand the scope of the review. Through the Review > Update [Your_Review] > Modify Review Checks wih... menu, you can import additional checks from your original checklist, or any other checklist of your choice. You can also use this menu action to re-import an existing check, for instance if that check has been redefined since you began you review. Re-importing an existing check causes any results (probes, ratings and notes) associated with that check to be lost.