Using CAD 'on demand' may be more effective than prompts

Radiologists' ability to detect malignant breast masses is more accurate when computer-aided detection (CAD) marks are hidden until they are needed, rather than displayed while the radiologist reads a study, according to new research from the Netherlands published online in Radiology.

Dutch researchers compared the effectiveness of an interactive CAD system -- in which CAD marks and their associated scores remain hidden unless their location is queried by the reader -- with the effect of traditional CAD prompts to detect malignant masses on digital mammography (Radiology, 22 October 2012).

"Most radiologists agree that CAD is helpful for detection of microcalcifications, for which [its] sensitivity is high," wrote lead author Rianne Hupse, of Radboud University Nijmegen Medical Center, and colleagues. "[But] there is less agreement regarding CAD's ability to find masses and architectural distortions. Many radiologists argue that CAD indicates too many false-positive results to have a beneficial effect on mass detection."

Using mammography CAD may lead to disappointing results because masses are missed due to incorrect interpretation, the authors wrote. Some research suggests that reader performance could be improved by combining reader scores with the presence and probability of CAD marks. This inspired Hupse's group to develop a CAD system that helped radiologists interpret suspicious regions, rather than helping with the initial detection of these regions.

"In our interactive system, CAD marks are only displayed 'on demand' for queried regions with a suspiciousness score," the group wrote. "Our system is aimed at screening. Instead of providing justification for the marks that are used in the existing CAD systems, our system relies on the reader for the initial detection process and aims to improve only interpretation and recall decisions."

For the study, nine screening radiologists and three residents trained in breast imaging read 200 cases twice, once with CAD prompts and once with interactive CAD. (The cases were acquired between 2003 and 2008, in a digital screening pilot project; Hupse's group selected 80 biopsy-proven cancer cases and 120 negative cases.) The CAD algorithm the researchers used was designed to detect malignant masses and architectural distortions, and was "trained" with 11,793 digitized film mammograms that contained 1,853 malignant mass regions.

In the prompting mode, all CAD regions were shown by a display of contours without scores that ranked their suspiciousness (although the software showed prompts when the suspicious area scores were above a particular threshold). In the interactive mode, regions that were defined by CAD remained hidden until the reader activated them with a mouse-click. Hupse's group assessed reader performance by using location receiver operator characteristics analysis.

Reader performance increased when CAD results were interactively displayed, and prompts had no major effect on reader performance, Hupse and colleagues found. Readers using CAD in the prompting mode had sensitivity of 84% (67 of 80 cases), while in the interactive mode they recorded sensitivity of 91% (73 of 80 cases).

"In the interactive mode, marks remained hidden unless regions that corresponded were probed," they wrote. "Because most radiologists probed only a limited number of regions, and only those that they were interested in, fewer false positives were displayed."

"CAD suspiciousness scores generally corresponded well with the observers' interpretations. When this was not the case, readers were alerted and paid more attention, which may have led to better decisions," the group concluded.