Renal Ultrasonography in the Evaluation of Acute Kidney Injury:  Developing a Risk Stratification Framework FREE

Acute kidney injury (AKI) is a common problem in hospitalized patients, with an incidence increasing from approximately 10 to 25 per 1000 discharges over the last 15 years.¹ Defined as an abrupt decline in renal function, indicated either by increased serum creatinine (CR) level (>0.3 mg/dL [to convert CR to micromoles per liter, multiply by 88.4] or 50% above baseline) or decreased urine production (<0.5 mL/kg/h over 6 hours), AKI is significantly associated with increased mortality.^2- 3 In the initial evaluation, a renal (or retroperitoneal) ultrasonography (RUS) study is often ordered to exclude an obstructive cause.^4- 5 If diagnosed, patients with urinary tract obstruction may require further interventions.

However, most cases of AKI are notcaused by obstruction.¹ In fact, hydronephrosis (HN), the evidence of obstruction on imaging, is only identified on RUS in 1% to 10% of patients with AKI.^6- 9 As a result, findings from RUS do not initiate a change in clinical management in the majority of patients with AKI who undergo the test. It has also been suggested that RUS might yield additional clinically useful information, yet little is known about how frequently this occurs.^8,10

Targeting RUS evaluation toward patients with a higher risk of HN would not only be clinically useful but could potentially conserve resources.¹¹ This type of patient-centered decision making and targeting of testing toward those most likely to benefit has been recently recommended by the Institute of Medicine and is a central component of comparative effectiveness research.¹²

Further information is needed to stratify patients with AKI according to the likelihood that RUS will yield clinically meaningful results. We sought to create a stratification system that would help clinicians ascertain the risk of renal obstruction among those with AKI. This approach would improve the pretest probability of a positive finding on RUS and hence the likelihood of influencing the management of patients most likely to benefit from intervention. Specifically, we designed and validated a decision rule that would identify those patients at low risk of obstruction, as well as those patients at low risk for an obstruction requiring surgical intervention. As a secondary analysis, we evaluated the additional value of RUS by assessing the presence of other non-HN but clinically useful findings. Finally, we assessed RUS use at Yale–New Haven Hospital (YNNH), New Haven, Connecticut, and the effectiveness of RUS screening in terms of number needed to screen (NNS).

We conducted a cross-sectional study of hospitalized patients with AKI using separate derivation and validation samples. A derivation sample was analyzed using the presence of HN on RUS as the dependent variable. Strata were created based on the presence of risk factors associated with HN. A validation sample was developed using all RUS studies performed over a 16-month period, and the presence of HN and HNRI was assessed in each risk group. This study was approved by the Yale Human Investigation Committee.

A 2-step approach was used to create samples of inpatients with AKI who undergo RUS. First, patients were identified by searching the YNNH imaging database for RUS studies performed on hospitalized adults (age >18 years) with suspected AKI from January 1, 2005, to May 1, 2009. Suspicion of AKI was defined by a list of terms we piloted in a 3-month analysis of RUS studies, including hydronephrosis, creatinine, and arf (for a complete list, see eAppendix). Only those patients who underwent RUS were included, rather than all patients with AKI, to construct an enriched sample of patients whose clinicians were concerned enough about an obstructive cause to order an RUS study.

Second, patients were excluded who did not meet the definition of AKI: a peak rise in serum CR level of at least 0.3 mg/dL from baseline during inpatient admission. Though AKI is a complex disorder with a changing definition, we operationalized it according to recent recommendations from the Acute Kidney Injury Network, whose members represent key societies in nephrology.³ Baseline serum CR level was defined as the lowest value in the 3 months prior to admission (if unavailable, then in the following order: lowest value 12 months prior to admission, baseline value described in the admission note, or lowest value during the current admission). Additional exclusion criteria included pregnancy, history of renal transplant, and previous diagnosis of HN within 30 days prior to RUS (considered follow-up studies, rather than primary diagnostic evaluations).

Separate derivation and validation samples were constructed from those eligible studies (Figure 1). In the derivation sample, 100 studies with HN on RUS and 100 randomly selected studies with normal findings were included. This approach was used to maximize power in the derivation sample, since HN is a relatively rare finding. In the validation sample, all eligible studies from January 1, 2008, to May 1, 2009, were included. In both derivation and validation samples, 1 study per patient was included.

We chose candidate risk factors based on clinical relevance and description in the salient medical literature.^7- 8,10 All data were abstracted from medical records (discharge summaries and clinical notes) by 4 trained reviewers. The abstraction form was piloted and refined on a sample of 50 patients. Interobserver agreement was calculated for 10% of the derivation sample across 36 total variables, each treated as an independent unit. The average proportion of identically abstracted variables between one reviewer (A.L.) and each of the other 3 reviewers was 95%. Medical chart reviewers were blinded to the RUS result for each patient. Demographic data included race, age, and sex. Clinical data were abstracted in 2 general categories: factors predisposing to obstructive AKI, such as history of abdominal or pelvic cancer, and factors making another more likely cause (eg, prerenal AKI). Other variables included laboratory data (eg, granular casts on urinalysis) or certain inpatient exposures (eg, radiographic contrast) (Table 1). Clinical variables were only coded if they were available and known by the clinical team prior to the maximum serum CR value and RUS date.

All data were constructed as categorical variables, except for the mean rise in CR level, age, and white blood cell count, which were constructed as continuous variables. Age and white blood cell count were subsequently dichotomized based on preliminary bivariate analysis. One variable was coded 2 ways. In our primary model “prerenal AKI” included history of sepsis or use of pressors during current admission, while in a second model (designed for sensitivity analysis), this variable also included history of hypotension prior to the onset of AKI, defined as at least 2 consecutive blood pressure measurements below 80 mm Hg systolic or below 60 mm Hg diastolic.

The study outcomes were HN and HN requiring intervention (HNRI). Any RUS report that described “hydronephrosis” in the findings section was considered an outcome event. HNRI was defined as a RUS-diagnosed HN followed by either placement of a urologic stent or nephrostomy tube after the RUS date. Incidental non-HN findings on RUS were defined as anatomic abnormalities or masses described in the RUS report (for a list of search terms, see eAppendix). To determine whether these findings were previously known to the clinical team, we searched for prior documentation in the medical record, dating back to the beginning of the patient's first encounter at YNHH.

The association between candidate risk factors and presence of HN on RUS was initially assessed for the derivation sample using bivariate logistic regression analysis. Clinically relevant candidate variables with a P value <.20 from the bivariate analysis were evaluated in a logistic regression model (benign prostatic hyperplasia was also included owing to its clinical significance; P = .38). Some clinically related variables were collapsed into single composite variables (Table 1). In multivariable logistic regression, variables were removed in order of decreasing P value, and the model's quality was assessed at each iteration using the Akaike information criteria (AIC) and C statistic. The AIC is used to compare models with a penalty for the number of variables evaluated, to balance the model's explanation of the data (ability to identify patients with HN) with its parsimony (number of variables).¹⁵ Stepwise regression was continued until the model's quality was optimized (according to the C statistic and AIC). We selected the most accurate model (ie, discrimination) and applied it to the validation sample. For a sensitivity analysis, we evaluated a second model, differing from the main model only in the definition of a single clinical variable (“prerenal status”), which showed poorer discrimination between patients with HN and patients without HN but had a lower AIC than the primary model.

A risk score was developed based on the individual odds ratio (OR) of each covariate. Because all but 1 risk factor had similar ORs, they were each awarded 1 risk point. Any patient with a history of HN was assigned a priori to the high-risk group, since these patients were considered most likely to need RUS in the setting of AKI. Also, this variable was the only one with an OR greatly different from the others (approximately 11, compared with 2-3). Using this scoring system, we segregated patients into 3 risk groups based on the prevalence of HN among patients with each risk score. This stratification was then applied to a validation sample. The sample size (N = 800) was calculated a priori and provided 80% power to detect a prevalence of HNRI in the low-risk group of 0.3% to 0.5%.

Finally, we calculated NNS to find 1 case of HN or HNRI for each risk group (ie, the total number of patients in each group divided by number of outcome events). To estimate cost associated with a positive finding, we multiplied the NNS by the approximate cost per study according to Medicare reimbursement.¹¹

Our derivation sample consisted of 100 patients with HN and 100 patients without HN. The mean age was 65.6 years, 56.5% were male, and 25.5% were black; none of these characteristics was significantly related to HN status (Table 1). Overall, patients with HN were more likely to have a previous diagnosis of HN (on RUS or abdominal/pelvic computerized tomography; 3% of patients without HN vs 28% of patients with HN; P < .001), a history of abdominal or pelvic cancer (14% vs 38%; P < .001), previous pelvic surgery (10% vs 19%; P = .11), a single functional kidney (1% vs 6%; P = .054), or hematuria (4% vs 13%; P = .02) during the selected admission. A history of urologic dilatation on imaging not considered HN, described as “pelvicaliectasis” or “caliectasis,” was not significant, nor was an imaging history of anatomic abnormalities, stones, masses, or cysts.

Conversely, patients with a normal RUS result were more likely to have granular casts on urinalysis (28% vs 18%; P = .09), a white blood cell count greater than 16 000/μL (48% vs 24%; P < .001); a history of congestive heart failure (22% vs 13%; P = .09); documented hypotension during the current admission (55% vs 39%; P = .01); or exposure to either aspirin (>81 mg/d), a diuretic or angiotensin-converting enzyme inhibitor, or intravenous vancomycin during the current admission (63% vs 39%; P = .001). Use of any intravenous antibiotic was also assessed but was not significant.

The final model was selected based on its high accuracy and low AIC score (Table 2). It consists of 7 variables (see Table 2 for item definitions): (1) history of HN (high-risk group); (2) recurrent urinary tract infections (1 point); (3) diagnosis consistent with possible obstruction (1 point); (4) nonblack race (1 point); and absence of the following: (5) exposure to inpatient nephrotoxic medications (1 point), (6) congestive heart failure (1 point), or (7) prerenal AKI (1 point). This system was applied to the derivation sample and the prevalence of HN was assessed for each risk score. Three distinct risk groups emerged: low (<2 points, 1%-20% prevalence of HN), medium (3 points, 20%-40% HN), and high (>3 points, >40% HN).

Our validation sample consisted of 797 patients (mean age, 65.6 years). Of these patients, 54.6% were male and 22.8% were black. Overall, 10.6% had HN, of which 31.7% required an intervention (3.3% of total sample).

Two models were used in this sample (Table 3). Our primary model 1 differed from model 2 only in its definition of prerenal status. It was more sensitive for HN but included fewer patients in the low-risk group (ie, less specific). Of 797 patients, 223 (27.8%) were assigned to the low-risk group, of whom 3.1% had HN (1 patient, or 0.4% [0.01%-2.5%] had HNRI). The prevalence of HN was 10.7% in the middle-risk group and 16.1% in the high-risk group (Figure 2). When dichotomized to low risk vs all others, the negative predictive value (NPV) of the stratification was 96.9% (95% confidence interval [CI], 95.7%-98.1%) for HN and 91.8% (95% CI, 89.9%-93.7%) sensitive, with a negative likelihood ratio (NLR) of 0.27. When the outcome was HNRI, the NPV increased to 99.6% (95% CI, 99.1%-100%) and the sensitivity increased to 96.3% (95% CI, 94.9%-97.6%), with an NLR of 0.13.

We assessed the presence of incidental findings on RUS in the entire validation sample. Among the 797 patients, there were 8 incidental findings (1%) unknown to the clinical team: 2 horseshoe kidneys, 4 extrarenal pelvises, and 2 complex cysts. Of these, none were found in low-risk patients.

Model 2 was less sensitive but more specific for HN. In the low-risk group there were 331 patients (41.5%), 5.1% of whom had HN (1 patient [0.3%] had HNRI). The NPV for HN was 94.9% (95% CI, 93.3%-96.4%) (for HNRI, 99.7% [95% CI, 99.3%-100.1%]), with a sensitivity of 80.0% (95% CI, 77.2%-82.8%) (for HNRI, 96.3% [95% CI, 94.9%-97.6%]) and an NLR of 0.45 (0.09 for HNRI).

The NNS to find 1 case of HN in the low-risk group for model 1 was 32, compared with HNRI, which required an NNS of 223 (Figure 3). If no RUS studies were ordered for low-risk patients, models 1 and 2 would permit reduced RUS use of 27.8% and 41.5%, respectively. At YNNH in 2008, approximately 700 RUS studies were performed in the setting of AKI on adult inpatients who met our inclusion criteria; 30% of whom were in the low-risk group. At an approximate cost of $200 per study, a 30% reduction in RUS imaging would result in an annual savings of $42 000 at one institution. To find 1 case of HN in the low-risk group for model 1 costs $6371 per positive study result; for HNRI in the same model, the cost is $44 600 per positive study result.

In this retrospective study, we found that approximately 10% of patients who had RUS ordered in the setting of AKI had HN on RUS and 3% had HNRI. This prevalence estimate of HN is similar to that in previous studies.^6- 9 We also derived and validated a decision rule to stratify inpatients with AKI by HN risk. It is based entirely on common clinical information and can be easily applied. Our primary model was 91.8% sensitive and had an NPV of 96.9% for HN. To our knowledge, no such decision model exists in the literature.^7- 8,10

In addition to considering all cases of HN, we also assessed those requiring an intervention. These cases are most important to identify because they represent patients for whom RUS affected their management. We defined HNRI as an immediate surgical intervention most likely to be performed in the setting of HN, which 27 of the patients (31.8%) with HN received in our validation sample. Our primary model was highly sensitive for HNRI (96.3%), with an NPV of 99.6%. Each model had an NLR of near 0.1 for HNRI, a value that affects the pretest probability of a positive finding to a large degree.¹⁶ There were other interventions performed on the patients with HN, but these were less likely to be a direct result of finding obstruction on RUS: 60% of patients received a urology consult; 13%, Foley placement; 2%, cystoscopy; and 1%, lithotripsy. Our models stratified all but 1 of the 27 patients with HNRI as either medium or high risk, with an NPV approaching 100%. They each classified the same patient with HNRI as low risk—a woman admitted with urosepsis and a history of colectomy and nephrolithiasis, who was diagnosed as having hepatocellular carcinoma after the time of RUS (beyond our window of analysis, after RUS).

We also considered the value of RUS in terms of its ability to provide clinically useful information other than the presence of HN. Though the primary reason a clinician orders a RUS study is to identify an obstructive cause, there are other incidental findings that may be important to discover, including cysts, masses, or anatomic abnormalities. In our validation sample of nearly 800 patients, there were no previously unknown incidental findings found in low-risk patients. Other findings, such as size and echogenicity were not included because they were considered unrelated to inpatient management. For many patients, the test result was not important enough to even be noted in the medical record. In the low-risk groups of both models, there were 20 unique patients with HN, 19 of whom did not receive an intervention. In 5 of these 19 cases, there was no mention of the RUS result in the patient's progress notes. For another 7, the result was not mentioned in their discharge summaries.

The evaluation of AKI is a nuanced process, one that likely requires clinical judgment not currently captured by our model. We were not able to include all possible risk factors for obstruction, either to maintain model parsimony or because of a lack of statistical evidence, or both. For instance, oliguria, a seemingly intuitive indication of obstruction, was not associated with HN, perhaps because of the degree of obstruction necessary to have a significant effect on urine production (ie, a specific but insensitive marker). As with the 1 patient with HNRI in the low-risk group, there are likely risk factors not included in this model that should be incorporated into clinical decision making, such as the fractional excretion of sodium, that would likely add further discrimination to the model. Since as many as 40% of adult inpatients with AKI may be assigned to the low-risk group by our model, there is room for studies to be ordered for patients whose risk of obstruction is likely higher than calculated, while still markedly decreasing overuse. Patients who are at higher risk for obstruction according to this model should always receive sonographic evaluation, while most low-risk patients should not.

The cost savings we estimate are substantial. To find 1 case of HNRI, we estimate that approximately $45 000 needs to be spent on RUS studies. Moreover, this analysis was conservative. We did not take into account all the potential savings of improved RUS use, such as decreased length of stay. Because unnecessary diagnostic imaging contributes significantly to rising health care costs, this rule may lead to significant potential savings for health care institutions.^17- 19 Establishing evidence-based guidelines may also ameliorate the marked geographic variation in medical spending.^20- 22

One likely objection to the selective use of RUS may be that a RUS study would still need to be ordered, regardless of a patient's risk of obstruction, under certain circumstances. For instance, those patients whose serum creatinine continues to rise after intravenous hydration and removal of nephrotoxic medications. In these situations, a RUS evaluation should likely be undertaken. However, anecdotal experience suggests that many studies are not ordered in this setting after an attempt at management has been made, but rather as part of an initial workup of increased CR level. Indeed, for many of the patients included in our study, a RUS study was ordered at the first sign of AKI, often concurrently or prior to hydration or medication changes. If our decision rule, at the least, causes clinicians to delay ordering a RUS study until their patients' AKI persists in the face of medical management, the use of RUS will still be markedly improved.

Our study has limitations. The decision rule was derived and validated in a retrospective fashion in a single institution. Future refinement and validation of the model should be pursued in prospective studies in other populations. In addition, our study population represents only those patients who underwent an inpatient RUS. Therefore, our sample likely represents a population enriched with obstruction (ie, those patients at such low risk of obstruction who never underwent RUS were not included). Therefore, the inclusion of all patients admitted with AKI, not just those who underwent RUS, would likely skew our HN prevalence data downward (improving the NPV of our model). Finally, our cost analyses are preliminary and require future work to fully understand the costs of RUS use, as well as the use of our decision rule (eg, the cost of missing a case of severe HN).

We have derived and validated a decision rule to help clinicians identify those patients at low risk of obstruction on RUS, and specifically those patients whose RUS result will likely affect their management. If further refined and validated, these models may allow clinicians to both order RUS studies for those patients at high risk for obstruction and delay or decide not to order a study on the roughly one-third of patients at low risk. As more attention is paid to the cost-effectiveness of diagnostic modalities, prioritizing testing for those at higher risk of abnormal findings will result in more informed diagnostic approaches. For inpatients with AKI, directing RUS studies toward those at greater risk of obstruction will aid clinical decision making and decrease the cost of evaluation.

Correspondence: Cary P. Gross, MD, Department of Internal Medicine–Primary Care, Yale School of Medicine, PO Box 209093, New Haven, CT 06520-8025 (cary.gross@yale.edu).

Accepted for Publication: April 9, 2010.

Author Contributions: Dr Gross had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Licurse, Kim, Forman, Formica, Makarov, Parikh, and Gross. Acquisition of data: Licurse and Kim. Analysis and interpretation of data: Licurse, Kim, Dziura, Forman, Formica, Makarov, Parikh, and Gross. Drafting of the manuscript: Licurse. Critical revision of the manuscript for important intellectual content: Licurse, Kim, Dziura, Forman, Formica, Makarov, Parikh, and Gross. Statistical analysis: Licurse, Dziura, and Parikh. Obtained funding: Licurse and Forman. Administrative, technical, and material support: Licurse, Kim, and Forman. Study supervision: Kim, Forman, Formica, Makarov, and Gross.

Financial Disclosure: None reported.

Funding/Support: Dr Licurse received funding for this work from the Doris Duke Charitable Foundation.