From the Departments of Radiology (Drs Flobbe, Bosch, and van Engelshoven), Surgery (Drs Bosch, Beets, and von Meyenfeldt), and Clinical Epidemiology and Medical Technology Assessment (Dr Kessels), Maastricht University Hospital; and the Department of Epidemiology, Maastricht University (Dr Nelemans), Maastricht, the Netherlands. The authors have no relevant financial interest in this article.
The use of ultrasonography (US) in diagnostic breast imaging is increasing. Restricting US to subgroups of patients who benefit most would result in a more efficient and effective application. This study assessed the diagnostic value of US as an adjunct to mammography (MAM) and a clinical examination (CE) in the diagnosis of breast cancer and the feasibility of selecting subgroups of patients who benefit the most.
Between October 1, 1999, and August 1, 2000, all consecutive patients referred for breast imaging underwent additional US after MAM and a CE. Results were scored on a 5-point grading scale of increasing suggestion of malignancy. Pathologic results during 12 months of follow-up were used as the criterion standard. Receiver-operating characteristic curve analysis assessed the diagnostic value of US in the whole population and in subgroups of patients according to indication for referral and age.
A total of 3835 breasts were examined in 2020 patients, with a 6.3% prevalence of breast cancer. Breast US detected 8 extra malignancies and correctly downgraded 332 cases from a positive to a negative diagnosis (ie, from a suggested malignancy to no malignancy). Receiver-operating characteristic curves showed a significant improvement in diagnostic value by adding US to MAM and a CE (area under the curve for CE + MAM + US vs CE + MAM, 0.99 vs 0.95; P = .002). The diagnostic yield improved significantly in patients referred for palpable breast lumps (P = .004) or referred from the National Breast Cancer Screening Program (P = .05). Less pronounced was the value in patients referred for other symptoms or for follow-up of a prior breast malignancy. When breast imaging of the contralateral breast or of asymptomatic patients referred for reassurance or follow-up of a prior benign lesion was performed, the value of additional US remained undefined because of the few malignancies found.
The systematic application of breast US improved the overall diagnostic yield. The diagnostic value increased most in patients with palpable breast lumps and in patients referred with abnormal screening MAM results.
WITH A LIFETIME risk of 10%, breast cancer is a major public health problem for women in Western countries.1 Its diagnosis and treatment are creating a financial burden to the health care system.
In addition to mammography (MAM), ultrasonography (US) has become the most widely used imaging method in the diagnosis of breast disease, especially in patients with palpable masses and normal or inconclusive MAM results.2- 4 Numbers from our hospital show that the application of additional US has increased from 28% of patients referred for MAM in 19945 to more than 35% in 1999. This is partly because of wider availability and accessibility, broadening clinical indications, and the idea that physicians believe they are less likely to be blamed for doing more testing than for doing too little.6 As a result, many examinations are actually redundant and will lead not only to increasing health care costs, but also to unnecessary inconvenience and discomfort to the patient, an unnecessary workup because of false-positive results, and longer waiting lists.
To reduce this health care problem and to aim for more effective and efficient use, additional US should be restricted to those subgroups of patients who benefit most from it.
Several studies7- 12 have assessed the diagnostic value of clinically indicated breast US as an adjunct to MAM. Despite the fact that combining MAM with US resulted in a higher sensitivity, in 3 studies7,8,10 the increase in sensitivity was reached at the expense of a decrease in specificity. None of these studies have applied US systematically in consecutive patients to obtain evidence-based information on subgroups of patients with the highest diagnostic yield.
The aim of this prospective study was, therefore, to assess the diagnostic value of breast US as an adjunct to MAM and the clinical examination (CE) in a consecutive set of patients referred for diagnostic breast imaging. Furthermore, the feasibility of selecting subgroups of patients who benefit most from additional US was studied.
Between October 1, 1999, and August 1, 2000, all consecutive patients referred to our radiology department for diagnostic breast imaging underwent additional US after a CE and MAM.
Patients were informed about the study by mail preceding their scheduled visit and in the department before their imaging examination, after which informed consent forms were signed. Patients were excluded from the study when US could not be performed because of logistic reasons or when no informed consent was given. In the Netherlands, a system of collective health insurance makes health care available to the whole population; therefore, no patients were excluded from the study because of an inability to pay or lack of insurance.
Investigations were performed in fixed order during a single visit. Bilateral CEs, performed while the patients were in standing and sitting positions, were followed by standard craniocaudal and mediolateral oblique MAM examinations (Siemens Mammomat-2 unit/Kodak Min-R film screen combination), then followed by whole breast US (model ATL5000, 12.5-MHz linear array transducer; Philips Medical Systems, Best, the Netherlands). Conforming to clinical practice, all examinations were performed and interpreted with full knowledge of prior test results.
Tests were performed by a team of radiologists with more than 5 years' experience in diagnostic breast imaging. In one patient, all examinations were performed by the same radiologist. In addition, study procedures were monitored and registered by a resident who was present at all examinations of all patients.
All relevant patient characteristics and the description, size, and location of lesions found at the different diagnostic examinations were registered in a standardized fashion. The diagnosis, based on CE, MAM, and US, was scored separately per breast on a 5-point grading scale, with increasing suggestion of malignancy (1 indicates a normal result; 2, benign lesion; 3, probably benign lesion; 4, suggestive malignant lesion; and 5, malignant lesion [0 was an indeterminate result]), based on the breast imaging reporting and data system lexicon for MAM and under development for US by the American College of Radiology.13,14
In addition to these separate diagnoses, a combined diagnosis of CE
+ MAM was calculated by taking the maximum diagnostic score of these modalities. Finally, a final diagnosis based on CE + MAM + US was defined mathematically as the maximum score of the modalities in case of a benign or a malignant diagnosis. When scores differed by more than 1 grade of suggestion of malignancy or when either of the 2 modalities included a probably benign score, the overall diagnosis of the radiologist interpreting all examinations ruled. The additional diagnostic value of US was determined by comparing the accuracy of CE + MAM
+ US with the accuracy of CE + MAM.
Indeterminate CE results (score, 0), as in cases of mastopathy, were recoded as benign (score, 2), and indeterminate MAM or US results were recorded as probably benign (score, 3), for analysis.
The criterion standard for the presence or absence of breast cancer was determined by the pathologic results of core needle biopsies, excision biopsies, and other surgical interventions during a follow-up of 12 months. An additional 2 months was added, accounting for administrative routing of test results at the end of the follow-up period.
Pathologic results were retrieved from the hospital pathology department and the Dutch Network and National Database for Pathology, to which all Dutch hospital pathology departments are linked. Breast cancer status was considered negative when no pathologic condition was reported in either system.
Receiver-operating characteristic (ROC) curves were constructed for the imaging results of CE, CE + MAM, and CE + MAM + US. Areas under the curve (AUCs) were calculated as a measure of the diagnostic performance, and differences were calculated and tested according to the methods of Hanley and McNeil.15,16 Furthermore, ROC curves were analyzed for subgroups of patients according to indications for referral and age.
There were 2720 scheduled imaging examinations, of which 112 were canceled and 84 were excluded from the study because patients had been included on earlier visits. For 279 patients, additional US was not realized because of logistical reasons, and 225 patients refused to sign informed consent form. The patients excluded from the study had a comparable prevalence of breast cancer, age distribution, reason for referral, and imaging interpretation.
A total of 2020 consecutive patients were included in the study and underwent a CE, MAM, and US of both breasts. The population consisted of 2000 women and 20 men (mean age, 50.2 years; range, 16.8-90.3 years). Patients were referred by general practitioners (n = 1044), surgeons (n = 712), and other specialists (n = 264). Indications for patient referral were as follows: a palpable breast lump (n = 470); other breast symptoms, such as pain or skin or nipple abnormalities (n = 486); follow-up of a prior breast malignancy (n = 438); follow-up of prior benign breast disease (n = 152); MAM abnormalities detected through the National Breast Cancer Screening Program (n = 144); family history of breast cancer (n = 234); patient's anxiety (n = 13); and other asymptomatic reasons (n = 83). These last 3 indications for referral were combined as "reassurance."
One patient was included twice because both visits concerned different indications in different breasts. There were 182 patients who underwent a prior mastectomy because of a history of breast cancer in whom the contralateral breast was investigated; in 23 other patients, only 1 breast was examined for other reasons.
So, 3835 breasts were examined in 2020 patients.
After a follow-up of 12 months, 129 malignancies were found in 127 patients, leading to a prevalence of 6.3% (127/2020). Two patients had bilateral breast cancer. Thirteen malignancies were carcinomas in situ only, and 116 were invasive carcinomas. In 123 cases, malignancies were found as a result of the diagnostic imaging described. Six malignancies were detected after 3 (n = 1), 7 (n = 1), and 12 (n = 4) months of follow-up. In 3 cases, a malignancy was found during the radiological follow-up of benign calcifications (1 after 3 months and 2 after 12 months). Two patients presented with a new palpable lesion at 7 and 12 months after the initial breast imaging in this study, which reported normal reading results. Finally, in 1 patient, yearly radiological follow-up of a prior breast malignancy led to the detection of a new lesion after 12 months.
Using a cutoff between benign and probably benign results, thus defining scores 3, 4, and 5 as positive results and scores 1 and 2 as negative results, a CE detected 69.0% of all malignancies (89/129), MAM detected 82.9% (107/129), and US detected 87.6% (113/129). The specificity reached 92.5% (3616/3910) for a CE, 91.9% (3405/3706) for MAM, and 95.5% (3556/3723) for US.
Table 1 shows the test results of CE + MAM + US and CE + MAM categorized by the absence or presence of breast cancer. For CE + MAM, the sensitivity was 91.5% (118/129), the specificity was 87.0% (3226/3706), the positive predictive value was 19.7% (118/[118 + 480]), and the negative predictive value was 99.7% (3226/[11 + 3226]); for CE + MAM + US, the sensitivity was 96.9% (125/129), the specificity was 94.8% (3512/3706), the positive predictive value was 39.2% (125/[125 + 194]), and the negative predictive value was 99.9% (3512/[4 + 3512]).
Breast US detected 8 extra malignancies that were missed by the CE + MAM. A detailed description of these cases is given in Table 2. Furthermore, US correctly downgraded 332 cases from a positive to a negative diagnosis (ie, from a suggested malignancy to no malignancy). On the contrary, there were 46 false-positive US results, and in 1 case, US incorrectly downgraded the imaging result to a negative diagnosis.
Receiver-operating characteristic curves were constructed for the results of CE, CE + MAM, and CE + MAM + US for all 3835 cases (Figure 1). Comparing the AUCs, there was a significant increase in diagnostic performance by the application of additional US (AUC for CE
+ MAM + US vs CE + MAM, 0.99 vs 0.95; P = .002). Furthermore, the AUCs for CE + MAM and CE + MAM + US were significantly different from the AUC for CE alone (P<.001 for both).
Receiver-operating characteristic curves for clinical examination (CE), CE combined with mammography (MAM), and CE + MAM combined with ultrasonography (US) for all 3835 cases. The area under the curve for CE was 0.84 (95% confidence interval [CI], 0.79-0.89); for CE + MAM, 0.95 (95% CI, 0.93-0.98); and for CE + MAM + US, 0.99 (95% CI, 0.97-0.997). The SEs were 0.03, 0.01, and 0.06, respectively.
A total of 1228 patients were referred for the imaging of a single breast, and 792 were referred for the examination of both sides. Thus, of the 3835 breasts examined, breast imaging was done on referral in 2812 cases and as screening of the contralateral side in 1023 cases. In Table 3, the AUC of ROC curves for these groups shows that the additional diagnostic value of US was highest in the referred cases. In patients in whom breast imaging of the contralateral side was performed or in whom reassurance was the reason for referral, no additional diagnostic value was demonstrated for US (P value not applicable because of small numbers).
To study subgroups of patients who benefit most from additional US, the diagnostic value was calculated through ROC curves by indication for referral and age categories (Table 4). The diagnostic value improved significantly in patients with palpable breast lumps, most pronounced for those 50 years or younger, and in patients referred through the National Breast Cancer Screening Program.
In the patients referred for palpable lumps, US correctly downgraded 133 positive CE + MAM results. Furthermore, US detected 3 new malignancies at the cost of 11 false-positive results.
Of the 147 cases referred by the National Breast Cancer Screening Program, the results of 40 MAMs + CEs were negative. Ultrasonography did not detect new malignancies in this group, but correctly downgraded 29 positive MAM results to a negative diagnosis.
Additional US also increased the diagnostic value in patients referred for other symptoms, such as pain and nipple or skin abnormalities (1 new malignancy was found), and in patients referred for follow-up of a prior breast malignancy (2 new malignancies were found), although the difference in AUCs was not statistically significant.
In patients referred for follow-up of a prior benign abnormality, additional US detected 1 extra malignancy and confirmed 2 malignant diagnoses by CE + MAM (P value not applicable because of small numbers).
Finally, among the 654 cases referred for reassurance, because of fear or a family history of breast cancer, US missed the single malignancy in this group, which was detected by CE + MAM (P value not applicable because of small numbers). There were 12 false-positive US results, of which 4 were also false-positive MAM results.
For the 8 patients in whom only breast US detected a malignancy, the indications for referral were palpable breast lumps (n = 3), follow-up of a prior malignancy (n = 2), follow-up of a prior benign abnormality (n = 1), and breast symptoms (n = 1). In 1 patient, the cancer was found by chance, while the reason for referral was a palpable mass on the contralateral side.
In this outpatient population, systematic application of US as an adjunct to a CE and MAM significantly increased the diagnostic performance. The diagnostic gain of US was found in the detection of 8 extra malignancies that were missed by CE and MAM and by the downgrading of false-positive CE and MAM results (n = 332). Most of these were cysts or fibroadenoma, although not all of them were confirmed by pathologic features.2,4,17,18 On the other hand, diagnostic loss was reflected by 46 false-positive US results and 1 false-negative US result.
The final diagnosis of CE + MAM + US had a sensitivity of 96.9% and a specificity of 94.8%. Integrated diagnoses combining the results of MAM and US have been reported in several studies.2,5,7- 12,19 However, many studies fail to explain how this integrated diagnosis was obtained, and only a few studies describe consecutive sets of patients. Comparison with our data should, therefore, be done carefully, but lower sensitivities and slightly higher specificities are reported.5,12,19 This may be explained by their study design, in which US was only applied on clinical indication. The prevalence of breast cancer in this population was 6.3%, and is consistent with the results of these other studies5,12,19 in consecutive patients, reporting prevalences of 5.0% to 7.1%.
Despite the overall increase in diagnostic yield by systematic application, breast US should be restricted to subgroups of patients who benefit most to reduce the costs, time needed for diagnostic examinations, and harmful consequences of unnecessary examinations. According to our results, the application of breast US was most beneficial to patients referred for palpable breast lumps and patients with an abnormal screening MAM result. Applying these criteria to our population, US would be indicated in 30.4% (614/2020) of patients referred for diagnostic breast imaging.
In the literature, the presence of a palpable mass in young women is a well-described indication for breast US as an initial diagnostic imaging test3,4,18,20 or as an adjunct to MAM.12,19 While most studies are referring to women 35 years and younger, our results are applicable to women 50 years and younger. Ultrasonography detected additional malignancies in 8 patients, of whom 3 were referred because of a palpable mass and 1 was referred because of a palpable lump at the CE. According to clinical guidelines, all palpable masses should undergo cytological investigation, which would have detected these malignancies as well. However, US still has an important diagnostic role in these cases by completing the triple assessment, consisting of a CE, radiological imaging, and a cytological diagnosis, which facilitates treatment planning. Because a proportion of the cytological investigation results will be indeterminate, US results will be valuable.
To our knowledge, an abnormal screening MAM result has not been reported in the literature as an indication for additional US; however, palpable or nonpalpable inconclusive MAM abnormalities are discussed.12,18,19,21,22 In the patients referred from the National Breast Cancer Screening Program, US reduced the number of false-positive MAM results by downgrading a quarter of all cases positive for a malignancy.
The increase in diagnostic value was not significant in patients referred with breast symptoms or in those referred for the follow-up of a prior malignancy. Remarkably, in this last group, the largest absolute increase in AUC was found, but the few cases positive for a malignancy and the relatively high SEs prevent us from drawing any definitive conclusions.15 Furthermore, in the groups of patients referred for follow-up of a previous benign lesion or for reassurance, there were too few malignant cases for constructing ROC curves and no additional value for US could be demonstrated.
The role of breast US in screening asymptomatic patients is a controversial one.2,4,23,24 Our data show 5 malignancies in the patients without a clinical indication (Table 3), of which 4 were found in contralateral breasts and only 1 was found in the group of referrals for reassurance (Table 4), suggesting a poor diagnostic yield of breast imaging as a whole in these patients.
An alternative way for recoding indeterminate test results for analysis would be through the comparison of likelihood ratios for each level of suggestion of malignancy,25 and recoding them as the level that is most alike. Doing so, indeterminate MAM results would be recoded as negative findings, but our final conclusions would be similar. However, we believe that our applied method of recoding approaches clinical practice best, because indeterminate MAM results will always lead to additional testing. Furthermore, the breast imaging reporting and data system lexicon also treats indeterminate MAM results as positive.13
Recoding indeterminate CE results as positive results, as done for MAM and US, also would not affect the AUC (0.83 vs 0.84) or change our conclusions.
Many diagnostic studies use 1 end point for the follow-up of patients included over several months or years. As a consequence, these studies are biased in their definition of the criterion standard because the length of follow-up is different for each case. Therefore, in the present study, an equal follow-up of 12 months was used for all patients, extended for 2 months for administrative transactions of results at the end of this period. The period of 12 months was chosen to minimize the chance of detection of newly developed (interval) cancers and to maximize the chance of detection of the malignancies present. Nevertheless, when including the total range of follow-up from October 1, 1999, to October 1, 2001 (median, 19 months; range, 14-24 months), no additional false-negative imaging results were found. Despite the passive nature of the follow-up, we believe that the registration system used covered all pathologic conditions available for the whole study population.
The indications for breast imaging in this study were defined by the referring physicians, and symptoms could not always be confirmed by the specialist conducting the CE (eg, the palpation of a palpable lump). Because such situations reflect clinical practice, we decided to use the specified reason for referral in these cases rather than our findings at the CE. Associated is the fact that additional indications for US could be defined based on the results from CE or from MAM. However, by applying the set of criteria in daily practice, the advice to perform or withhold additional US could be given at the stage of referral for breast imaging. By doing so, the planning of breast imaging could be optimized and the number of redundant US examinations reduced. Additional research will be needed to determine the cost-effectiveness of the application of breast US in these specified groups of patients.
In conclusion, the systematic application of additional breast US improved the diagnostic performance in this population. Although the overall diagnostic yield improved, the number of unnecessary examinations can be reduced by restricting breast US mainly to patients with palpable breast lumps and patients referred with abnormal screening MAM results. The implementation of these evidence-based indications in clinical practice is expected to lead to more efficient and effective use of breast US.
Corresponding author and reprints: Karin Flobbe, PhD, Department of Radiology, Maastricht University Hospital, PO Box 5800, 6202 AZ Maastricht, the Netherlands (e-mail: email@example.com).
Accepted for publication August 15, 2002.
This study was supported by the Dutch Health Care Insurance Board, Amstelveen, the Netherlands.
We thank Kristel Vranken, MD, Dick Koster, MD, Aernout Borstlap, MD, and Ed van der Linden, MD, for their effort in the interpretation of the imaging examination results; Peggy Habets for data entry; and Mariëlle Casparie, from the Dutch Network and National Database for Pathology, for supplying us with pathological data.
Thank you for submitting a comment on this article. It will be reviewed by JAMA Internal Medicine editors. You will be notified when your comment has been published. Comments should not exceed 500 words of text and 10 references.
Do not submit personal medical questions or information that could identify a specific patient, questions about a particular case, or general inquiries to an author. Only content that has not been published, posted, or submitted elsewhere should be submitted. By submitting this Comment, you and any coauthors transfer copyright to the journal if your Comment is posted.
* = Required Field
Disclosure of Any Conflicts of Interest*
Indicate all relevant conflicts of interest of each author below, including all relevant financial interests, activities, and relationships within the past 3 years including, but not limited to, employment, affiliation, grants or funding, consultancies, honoraria or payment, speakers’ bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued. If all authors have none, check "No potential conflicts or relevant financial interests" in the box below. Please also indicate any funding received in support of this work. The information will be posted with your response.
Register and get free email Table of Contents alerts, saved searches, PowerPoint downloads, CME quizzes, and more
Subscribe for full-text access to content from 1998 forward and a host of useful features
Activate your current subscription (AMA members and current subscribers)
Purchase Online Access to this article for 24 hours
Some tools below are only available to our subscribers or users with an online account.
Download citation file:
Web of Science® Times Cited: 34
Customize your page view by dragging & repositioning the boxes below.
The Rational Clinical Examination
Evidence Summary and Review 2
The Rational Clinical Examination
Location of Paracentesis and Ultrasound Guidance
All results at
and access these and other features:
Enter your username and email address. We'll send you a link to reset your password.
Enter your username and email address. We'll send instructions on how to reset your password to the email address we have on record.
Athens and Shibboleth are access management services that provide single sign-on to protected resources. They replace the multiple user names and passwords necessary to access subscription-based content with a single user name and password that can be entered once per session. It operates independently of a user's location or IP address. If your institution uses Athens or Shibboleth authentication, please contact your site administrator to receive your user name and password.