|
Malaysian Journal of Medical Sciences, Vol. 11, No. 2, July 2004, pp. 34-40 ORIGINAL ARTICLE RELIABILITY AND VALIDITY OF THE RESPIRATORY SCORE IN THE ASSESSMENT OF ACUTE BRONCHIOLITIS Ho Jen Chin & Quah Ban Seng Department of Pediatrics,
School of Medical Sciences, Universiti Sains Malaysia, Health Campus,
16150 Kubang Kerian, Kelantan, Malaysia Submitted-12.5.2003, Code Number: mj04018 ABSTRACT Children with bronchiolitis from Hospitals Universiti Sains Malaysia (HUSM) and Hospital Kota Bharu (HKB) were student using the Kristjansson Respiratory Score and the Wang Respiratory Score respectively. Saturation of oxygen (SaO2) was measured with a pulse-oximeter while the child is breathing room-air. Two observers assessed the respiratory scores in all children independently. The correlation between respiratory scores and SaO2 was assessed using Spearman’s Rho, and the inter-rater reliability of respiratory scores determined using intraclasscorelation coefficient. There were 29 children in HUSM and 25 from HKB with a median age of 8 months (IQR 4.5 months) and 9 months(IQR 7 months) respectively. In HUSM, the median Kristjansson Respiratory Score for both observers was 4 (IQR 2), and the median SaO2 was 96% (IQR 3%). The correlation coefficient between the Kristjansson Respiratory Score and SaO2 for the first observer was –0.75 (p <0.001), and for the second observer –0.73 (p <0.001). In HKB, the median Wang Respiratory Score was also similar for both observers (median 4 IQR 4.5), and the median (IQR) for SaO2 was 96% (2%). The correlation coefficient between the Wang Respiratory Score and SaO2 for the first observer was –0.41 (p = 0.04) and for the second observer –0.43 (p = 0.03). The inter-rater reliability between the first and second observer was high for both the Kristjansson Respiratory (ICC 0.89) and the Wang Respiratory Scores (ICC 0.99). In conclusion the validity of the Kristjansson Respiratory Score was high whereas the validity of the Wang Respiratory Score was moderate in the assessment of the severity of acute bronchiolitis. Both respiratory scores and physical signs showed high agreement between observers. The Kristjansson Respiratory Score should be considered for use by medical personnel in the assessment of the severity of acute bronchiolitis in children. Key words : bronchiolitis, respiratory scores, validity, reliability Introduction Bronchiolitis is a clinical syndrome characterised by pulmonary hyperinflation and wheezing in association with a viral lower respiratory infection (1). It is an acute, highly communicable lower respiratory tract infection, which is the most common serious illness in childhood and the most common reason for hospitalisation of infants beyond the neonatal period. Its morbidity is high, accounting for 17% of hospitalisations in infants’ (9 admissions per 1000 child-year) in New York State (2). The assessment of the severity of bronchiolitis is necessary not only in daily practice but also in clinical trials. Children with severe bronchiolitis are at risk for severe hypoxaemia, and in daily practice paediatricians are often faced with the task of deciding which children require hospitalisation so as to avoid unnecessary morbidity and mortality. As pulmonary functions are difficult to perform in infants, respiratory scores are often used to evaluate the efficacy of medications for bronchiolitis. Various respiratory scores have been used to evaluate the response of acute bronchiolitis towards many types of medication such as epinephrine (3), nebulised albuterol (4), nebulised metaproterenol (5), nebulised racemic adrenaline (6) and other bronchodilators (7). Respiratory scores consisting of a combination of clinical symptoms and physical signs, are frequently used to estimate the severity of acute lower airway infection in children. Reports on the reliability of such signs have shown considerable disagreement among physicians, leading to unreliable clinical observations in respiratory diseases. Therefore, it is important to investigate the inter-observer agreement on the presence or absence of physical signs in acute bronchiolitis in order to derive a consistent diagnosis from different examiners irrespective of the situation. A search of the medical literature (19661992) resulted in the identification of no less than 16 clinical asthma scores (8). However, most scores were designed ad hoc based on clinical experience and face validity, and lacked information on reliability and validity(8). In this study we chose to evaluate two respiratory scores, the Kristjansson Respiratory Score (6) and the Wang Respiratory Score (9) for the assessment of the severity of acute bronchiolitis because the physical signs used in these two respiratory scores are simple and well categorised. The first objective of this study was to examine the validity of both the Kristjansson Respiratory Score and the Wang Respiratory Score by correlating the scores with oxygen saturation in room air in children with acute bronchiolitis. The second objective was to determine the inter-rater reliability of the above clinical respiratory scores in the assessment of the severity of acute bronchiolitis. MethodologyChildren admitted to Hospital USM (HUSM) from June 2000 to July 2001 and Hospital Kota Bharu (HKB) from July 2000 to June 2001 for acute bronchiolitis were recruited for the study. Children aged six to eighteen months who were admitted for the first episode of acute wheezing or rhonchi, tachypnoea, and chest retraction, preceded by or associated with cough, coryza, rhinorrhoea and had an axillary temperature >37.5ºC were eligible for the study. Children were excluded from the study if they had: i) an underlying disease that might affect the cardiopulmonary status (e.g. bronchopulmonary dysplasia, prematurity, assisted ventilation during the neonatal period, congenital heart disease or immunodeficiency); ii) asthma diagnosed by a physician, iii) wheezing or cough that had previously been treated with bronchodilators or corticosteroids within the preceding 2 weeks; or iv) had been treated with 35% or more of inspired oxygen by head box or equivalent during the neonatal period. Children admitted in HUSM were assessed with the Kristjansson Respiratory Score (6) whereas those admitted in HKB were assessed with the Wang Respiratory Score (9). The Kristjansson Respiratory Score consists of five clinical signs (Table 1) with scores for each sign ranging from zero to two. The possible total scores ranged from 0-10. Whistling expiratory sounds heard with and without the stethoscope was considered as rhonchi and wheeze respectively. The 4-item Wang Respiratory Score consists of respiratory rate, wheezing, chest retraction and general condition (Table 2) and each clinical sign was scored from zero to 3 except for the general condition, which was scored zero for normal and 3 for irritability or lethargy. The possible total score ranged from 0-12. The respiratory rate was determined by counting the respiration for 30 seconds in both respiratory scores. Saturation of oxygen (SaO2), which was the gold standard for severity of acute bronchiolitis, was measured with a pulse-oximeter (Nellor Puriton Bennett NPB-295 in HUSM and MSA Medical Product, Pittsburgh, 8000 in HKB) while the child was breathing room-air. The maximum SaO2 was read for each child after the pulse-oximeter had been attached to the patient for at least five minutes. In children who were given bronchodilators at the casualty, SaO2 was assessed 1 hour after treatment. Two observers independently assessed all children on both respiratory scores. The first observer was the main investigator and the second observer was either a Paediatrician, Paediatric Medical Officer, House Officer, or Medical Student (fourth or fifth year). All observers were briefed on the scoring system of each respiratory score and the purpose of the study. All observations were taken when the child was awake and not crying. Approval for the study was obtained from the Hospital Director of both hospitals and the Research and Ethics Committee, School of Medical Sciences, Universiti Sains Malaysia. A verbal consent was obtained from parents before the enrolment of children in the study. The data was analysed using SPSS for Windows version 9.0. The validity of the respiratory scores was assessed by correlating total respiratory scores of the main investigator with SaO2 using Spearman’s Rho. A sample size of thirty was estimated to detect a correlation coefficient of 0.5 at an alpha error of 5% and beta error of 20%. The inter-rater reliability of the scores (between first and second observer) was determined using a weighted kappa for the ordinal variables and intraclasscorrelation coefficient (ICC) for the continuous variables. A p value of less than 0.05 was considered statistically significant. ResultsThere were 29 children from HUSM and 25 from HKB with a median age of 8 months (IQR 4.5 months) and 9 months(IQR 7 months) respectively. There were 19 (66%) boys and 15 boys (60%) in HUSM and HKB respectively. In HUSM there were 28 (97%) Malays, but in HKB all children were Malays. In HUSM, the median (IQR) Kristjansson Respiratory Score for both observers was 4 (2), and the median (IQR) for SaO2 was 96% (3%). There was a significant negative correlation between the Kristjansson Respiratory Score and SaO2 for both the first observer (r = –0.75, p <0.001) and second observer (r= –0.73, (p <0.001) (Figure 1). In HKB, the median (IQR) Wang Respiratory Score for both observers was 4 (4.5), and the median (IQR) for SaO2 was 96% (2%). There was also a significant correlation between the Wang Respiratory Score and SaO2 both the first observer (r= –0.41, p = 0.04) and second observer (r= –0.43 p = 0.03) (Figure 2). However the magnitude of the correlation was less than the Kristjansson Respiratory Score. Among individual components of the Kristjansson Respiratory Score (Table 3) general condition had the strongest negative correlation with SaO2 for both observers. For the first observer, respiratory rate had the lowest correlation with SaO2 but for the second observer, skin colour had the lowest correlation with SaO2. Among individual components of the Wang Respiratory Score (Table 4) general condition also showed the highest correlation with SaO2 for both observers. However respiratory rate had the lowest correlation with SaO2 in both observers. The inter-rater reliability between the first and second observer was high for the Kristjansson Respiratory (ICC 0.89). The inter-rater liability for individual clinical signs Kristjansson Respiratory Score also high with weighted kappa ranging from 0.79 to 0.91 (Table 5). Respiratory rate showed the highest reliability with kappa value of 0.91, followed by general condition (0.88), chest recession (0.84), breath sound (0.81) and skin colour (0.79). The inter-rater reliability of the Wang Respiratory Score between the first and second observer was high (ICC 0.99). Individual components of the respiratory score also showed high inter-rater-raliability with weighted kappa ranging from 1.00 to 0.90. Among the four physical signs, wheezing and general condition had the highest reliability (weighted kappa of 1.00). All the components in the Wang Respiratory Score showed higher inter-rater reliability compared to those of Kristjansson Respiratory Score. DiscussionIn this study, both Kristjansson Respiratory Score and Wang Respiratory Score correlated significantly with SaO2. However, the magnitude of correlation for the Kristjansson Respiratory Score was higher than the Wang Respiratory Score. The SaO2 determined by pulse oximeter is the single best objective predictor of severity in infants with acute bronchiolitis (10, 11) and was chosen as the gold standard in this study. The severity of acute bronchiolitis is closely related to the degree of hypoxaemia and hypercapnoea arising from an abnormal distribution of ventilation relative to perfusion. Thus both respiratory scores comprised of clinical signs which represent the degree of hypoxaemia. The average negative correlation between oximeter and the two respiratory scores was similarly observed by Hal et al (12). In both respiratory scores, general condition had the highest correlation with SaO2. The more detailed evaluation of general condition in the Kristjansson Respiratory Score resulted in a higher validity than that in the Wang Respiratory Score. Shaw et al. (10) showed that a simple clinical description of the infants’general appearance (“well appearance”, “ill –not toxic”or “toxic”) by an attending physician was a helpful measure in assessing the degree of illness in children with acute bronchiolitis. In the Kristjansson Respiratory Score, chest recession also showed a significant correlation with the saturation of oxygen. It was postulated that chest recession reflects the effort to improve oxygenation in hypoxaemic children with lower respiratory infection and the use of accessory muscles had been shown to correlate strongly with the severity of disease (10). In this study, chest retraction in the Wang Respiratory Score showed a poorer correlation with SaO2 than chest recession in the Kristjansson Respiratory Score. A simple description of chest recession such as none, moderate and severe as in Kristjansson Respiratory Score has been shown to be more valid than the more detailed description of chest retraction in the Wang Respiratory score. In both respiratory scores respiratory rate and breath sounds (wheezing) only had an average or non-significant correlation with SaO2. Children with acute bronchiolitis may be clinically well even when the inspiratory and expiratory airway resistance is high (13), because increased resistance is due to dynamic narrowing of the airways. Thus, the respiratory rate and breath sounds may not correlate well with SaO2 and are not good indicators of disease severity. This finding was similarly observed by Hall et al (12) who showed a poor correlation of respiratory rate and disease severity. Skin colour was included in the Kristjansson Respiratory Score but not the Wang Respiratory Score. However the correlation with saturation of oxygen was poor in both observers. The result in this study concurs with the findings of Shaw et al (10) and Mulholland et al (11). This is not surprising as skin colour is affected by many factors besides peripheral tissue perfusion and oxygenation. Cyanosis which only occurs in children with severe hypoxaemia may not be easily detectable in children with anaemia and dark pigmentation. In summary this study showed that general condition and chest recession were better predictors of severity of acute bronchiolitis compared with respiratory rate, breath sounds and wheezing Both the Kristjansson Respiratory Score and Wang Respiratory Score showed high inter-rater reliability, with the Wang Respiratory Score having a slightly reliability than the Kristjansson Respiratory Score. This difference is probably due to the simplicity of the Wang Respiratory Score, especially the classification of general condition which has only two scores. In the Kristjansson Respiratory Score, skin colour had the lowest reliability. This may be due to the difficulty in assessing skin colour which is also affected by anaemia and pigmentation of children in this region. In the Wang Respiratory Score, general condition and wheezing had the highest reliability. This is most probably due to the simplicity of classification of these two signs. Physical signs, which can be objectively evaluated, have been shown to have a higher inter-rater agreement than subjective physical signs (14, 15). Spiteri et al (16) showed that the reliability of recognising physical signs in the examination of the chest among 24 physicians was poor. Wang et al (15) evaluated the inter-rater agreement for respiratory signs in infants hospitalised with lower respiratory infections, and showed that the agreement beyond chance were all below 50%. In these studies, poor inter-rater agreement was associated with poorly defined and poorly categorised physical signs. The variation of clinical experiences among different observers could be also affecting the reliability of a score. Even though two different brands of pulse oximeter were used in the two hospitals, each score was evaluated against a single brand of pulse oximeter. As the readings of each pulse oximeter were not expected to differ greatly from each other, the validity of both respiratory scores may be compared. Environmental factors such as room temperature and lighting, and physical conditions such as hypothermia and hypotension are not expected to affect the oximeter reading as the environment of the hospitals are not unduly different and the severity of bronchiolitis of children admitted in both hospitals is not markedly different. Despite the differences in clinical experiences among the observers in this study the reliability of the respiratory scores was high. In conclusion the validity of the Kristjansson Respiratory Score was high whereas the validity of the Wang Respiratory Score was moderate in the assessment of the severity of acute bronchiolitis. In the Kristjansson Respiratory Score, general condition and chest retraction were the best indicators of the severity of disease. Both respiratory scores and physical signs showed high agreement between observers. Between the two respiratory scores the Kristjansson Respiratory Score is more valid and reliable and should be considered for use by medical personnel in the assessment of the severity of acute bronchiolitis. References
© Copyright 2004 - Malaysian Journal of Medical Science The following images related to this document are available:Photo images[mj04018t4.jpg] [mj04018f2.jpg] [mj04018t3.jpg] [mj04018t2.jpg] [mj04018t5.jpg] [mj04018t1.jpg] [mj04018f1.jpg] |
|