Reliability of the general severity scoring systems, APACHE II and SAPS II

To investigate the reproducibility of hospital mortality as predicted by two general severity scoring systems, APACHE II and SAPS II, data from patients consecutively admitted to the intensive care unit (ICU) during 1994 were analysed.

Abstract

Objective: To investigate the reproducibility of hospital mortality as predicted by two general severity scoring systems, APACHE II and SAPS II. Design: Data from patients consecutively admitted to the intensive care unit (ICU) during 1994 were analysed. Resident doctors abstracted and recorded the values of the APACHE II variables. The APACHE II score and the probability of hospital mortality of each patient were recorded in a database made up of 405 cases. Medical records of 81 cases selected from this database were reviewed by four 'new observers'. Two of them abstracted values of the APACHE II components and the others those of the SAPS II score. Comparison was performed on the data from the 'new observers' (APACHE II and SAPS II) and the 'old abstractions'(APACHE II). Setting: Two four-bedded multi-disciplinary ICUs of a 1,100-bed teaching hospital. Results: The new observers recorded all data except the blood gas analyses from four patients which were not available. Considering 'old abstractions' of the APACHE II components, 79 items of information were not recorded. Depending on the 'new observer', six or seven of these were outside the normal range. Intraclass correlation coefficient (ICC) for values and scores of APACHE II variables were generally high. Concordance on diagnostic category was very poor when 'old' and 'new' abstractions were compared. For SAPS II, the ICC for scores of white blood cell count and serum sodium appeared much lower compared to those for values. The 95% confidence interval (CI) limits of agreement for APACHE II mortality predictions ranged from -11% to 9% for the two new observers, with seven cases beyond these limits. They ranged from -18% to 20% and from -20% to 19%, considering each 'new observer' and previous 'old abstractions', with seven and eight cases beyond the limits, respectively. The 95% CI limits of agreement for SAPS II ranged from -21% to 18% for the two new observers, with eight cases outside the limits. There was no statistically significant difference in probability of hospital mortality calculated using values of APACHE II components abstracted by the different observers. The same was observed when SAPS II variables were considered. Conclusions: Most of the data not recorded by observers during abstraction of values of APACHE II variables were within the normal range. The diagnostic category of the APACHE II score does not show good reproducibility. Small differences in values of some SAPS II variables between observers determine important differences in scores and probability of hospital mortality. Hospital mortality rates predicted using APACHE II or SAPS II in groups of patients show good reproducibility.