False Positives and False Negatives in Dataflow Analysis

Dataflow analysis does not always yield a precise result. Analysis limitations may mean that conditions which should be identified will sometimes go unreported ('false negatives') and conditions which are benign may give rise to an unnecessary diagnostic ('false positives').

A particular difficulty in dataflow analysis is that it is often impossible to confirm or dismiss a potential problem with certainty. For example, when investigating a danger, such as a NULL pointer dereference, dataflow analysis can arrive at one of three conclusions:

• This pointer will always be NULL.
• This pointer will never be NULL.
• This pointer will could be NULL.
• This pointer will sometimes be NULL (i.e. on some but not all iterations of a loop)
• This pointer will be NULL if a particular path elsewhere is executed
• This pointer will be NULL if a loop construct elsewhere is bypassedd

Dataflow analysis is a technique which has inherent limitations; there will always be run-time problems which dataflow analysis will not be able to diagnose, either because there is insufficient information available within the source code or because the analysis becomes too complex. Judicious simplifications and compromises have to be made in order to perform analysis within reasonable constraints of time and resources.

False positive diagnostics are sometimes generated when dataflow analysis is not aware that a function call does not 'return'. Functions such as exit() and abort() will automatically be recognized as 'non-returning'; any other 'non-returning' function may be declared within program code using a #pragma directive of the form:

    #pragma PRQA_NO_RETURN funcname

The incidence of false positives will usually be low among definite and apparent diagnostics but much more frequent among suspicious and apparent messages. Suspicious and apparent messages exist to warn about an issue when circumstances are intrinsically uncertain. Unless code has been written defensively with a high level of security in mind, possible messages in particular may be of limited usefulness.

Index