African Health Sciences Makerere University Medical School ISSN: 1680-6905 EISSN: 1729-0503 Vol. 9, Num. 3, 2009, pp. 186-192

African Health Sciences, Vol. 9, No. 3, Sept, 2009, pp. 186-192

A preliminary examination of the construct validity of the KABC-II in Ugandan children with a history of cerebral malaria

^a,bBangirana P*, ^aSeggane-Musisi, ^bAllebeck P, ^cGiordani B, ^dJohn CC , ^eOpoka OR, ^eByarugaba J, ^f,gEhnvall A, ^c,hBoivin MJ

^a Department of Psychiatry, Makerere University School of Medicine, Kampala, Uganda
^b Department of Public Health Sciences, Karolinska Institutet, Stockholm, Sweden
^c Neuropsychology Section, Department of Psychiatry, University of Michigan, Ann Arbor, MI, USA
^d Department of Pediatrics, University of Minnesota, Minneapolis, MN, USA
^e Department of Pediatrics and Child Health, Makerere University School of Medicine, Kampala, Uganda
^f Institute of Clinical Neurosciences, Department of Psychiatry, Gothenburg University, Gothenburg, Sweden
^g Center for Cognitive Therapy, Gothenburg, Sweden
^h International Neurologic and Psychiatric Epidemiology Program, Michigan State University, East Lansing, MI, USA
*Corresponding author: Paul Bangirana, Makerere University School of Medicine, Department of Psychiatry, P. O. Box 7072, Kampala, Uganda. Email: pbangirana@yahoo.com Tel: +256-772-673831

Code Number: hs09041

Abstract

Background: Several diseases and adverse conditions affect the cognitive development of children in Sub-Saharan African. There is need to assess these children to determine which abilities are affected and the severity of the damage so as to plan interventions accordingly. However most psychological tests developed in the West have not been validated in this region making it impossible to know whether they measure what they were intended to in African children.
Objective: To examine the construct validity of the Kaufman Assessment Battery for Children, Second Edition (KABC-II) in Ugandan children.
Methods: Sixty five Ugandan children aged 7 to 16 years (Mean=9.90, SD=2.46) were tested using the KABC-II 44.59 months (SD=2.82) after an episode of cerebral malaria. The KABC-II scales of Sequential Processing, Simultaneous Processing, Planning and Learning were administered. In order to identify which factors result from administering the KABC-II in these children, factor analysis using principal component analysis with Varimax rotation was applied to the subtests making up the above scales.
Results: Five factors emerged after factor analysis comprising of subtests measuring Sequential Processing, Simultaneous Processing, Planning and Learning. The fifth scale comprised of subtests measuring immediate and delayed recall.
Conclusion: This preliminary study in Ugandan children shows the KABC-II to have good construct validity with subtests measuring similar abilities loading on the same factor. The KABC-II can be used in assessing Ugandan children after a few modifications. Further analysis of its psychometric properties in Ugandan children is required.

Key Words: neuropsychology, cross-cultural, Africa, children, validation

Introduction

It is estimated that 780 million children worldwide have cognitive deficits with the majority of them living in low income countries¹. Stunting, inadequate cognitive stimulation, iodine deficiency and iron deficiency anemia have been identified as the four main risk factors for these cognitive limitations². Others include infectious diseases like malaria and HIV, intestinal parasites, intrauterine growth retardation, maternal depression, exposure to violence, exposure to heavy metals and famine^1,2. Sub-Saharan Africa has the highest number of children at risk of cognitive deficits with 61% of its children of less than five years being stunted, living in poverty or both³.

Studies in Africa that have looked at cognitive functioning in children faced with some of the above risk factors have shown deficits in attention, memory, language, visual spatial skills and executive functions^4-11. These studies using tests developed in the West have given valuable information on the cognitive outcomes in at risk children. However there are questions whether these tests are measuring what they are intended to measure in African children^10,12. It is important to validate these tests or to develop appropriate tests to accurately determine the frequency of cognitive impairment in African children. Such appropriate tests may help convince relevant authorities for funds to carry out interventions and to assess the effect of these interventions^12,13.

The Kaufmann Assessment Battery for Children (KABC) ¹⁴ is one Western measure of cognition that has been widely used in assessing the effects of cerebral malaria (CM) in African children^5,8,15,18. Validation studies of the KABC in Africa and Asia showed it to retain its construct validity and to be sensitive to socioeconomic indicators, disease effects and tactile learning problems^16,19,21. Also, a meta-analysis of KABC validation studies across cultures has supported the factor integrity of the Sequential Processing versus Simultaneous Processing distinction as originally intended in the design of the KABC²².

The Kaufmann Assessment Battery for Children, second edition (KABC-II) was published recently with a wider age and measuring more abilities²³. Whereas the original KABC was grounded in the Cattell-Horn theory of crystallized versus fluid intelligence, the KABC-II is based on both the more current Cattell-Horn-Carroll psychometric model of broad and narrow abilities and Luria's neuropsychological theory of information processing²³. This change enabled the KABC-II to assess the same abilities as the original KABC as well as two new abilities; Planning and Learning. The KABC-II still retains the nonverbal global scale capabilities and the capacity for culture fairness and cross-cultural adaptability that was the hallmark of its predecessor. Even with these improvements, it is necessary to examine the psychometric properties of this new test prior to its use in African children.

The aim of this study was to assess the construct validity of the KABC-II in the Ugandan context. Particular interest was paid to Luria's model of information processing comprising of four scales; Sequential Processing, Simultaneous Processing, Learning and Planning²³. The research question to be answered was; does assessment of information processing in Ugandan brain injured children using the KABC-II result in the above four scales? By using children with a history of cerebral insult, this study will examine one of KABC-II's purposes; to understand the brain-behaviour relationships in individuals with brain dysfunction or damage²³. This will help to determine whether the KABC-II measures information processing as it was intended to measure when used with Ugandan children with brain injury.

Methods

Study population and recruitment

The present study was conducted at Mulago Hospital in Kampala, Uganda. Respondents were part of a study piloting cognitive rehabilitation interventions for children who had cerebral malaria (CM). Full details of this study cohort, including enrollment criteria, and the baseline demographic, clinical and laboratory findings during acute CM illness, have been published elsewhere^15,17. Briefly, children 4 to 12 years of age were enrolled in the original study if they were admitted to Mulago Hospital and met the WHO criteria for CM: coma (Blantyre coma scale less or equal to 2 or Glasgow coma scale less or equal to 8), P. falciparum on a blood smear, and no other cause for coma. Lumbar punctures were performed to rule out other causes of meningitis and encephalitis. Exclusion criteria at enrollment included: (a) a past history of meningitis, encephalitis or any brain disorder, including cerebral malaria or repeated seizures (b) a history of developmental delay (c) prior admission for malnutrition (d) a history of any chronic illness.

A total of 86 children were initially recruited. Five children were excluded because of prior severe illness (2) or misdiagnosis; meningitis (1), rabies(1), coma score not correctly calculated (1), and 2 died during the admission. Home environment was assessed using a version of the Middle Childhood HOME inventory²⁴ adapted for Uganda. The HOME inventory assesses the stimulation and learning opportunities offered by the child's home environment. It has 59 items however item 40 `Family member has taken child to (or arranged for child to visit) a scientific, historical or art museum within past year' was omitted because it was deemed not applicable to most of the children in the sample thus leaving 58 items for use in the study. Nutrition was assessed by comparing weight for age to published norms²⁵ and obtaining a standardized z-score (Epi Info 3.3.2; Centers for Disease Control and Prevention, Atlanta, GA). Level of education of the child was scored as follows: None = 0, Nursery = 1, Primary school grades 1-7 = 2-8, Secondary education = 9.

In previous studies of this cohort, cognitive testing was performed at discharge, then 3 months, 6 months and at 24 months after discharge as previously reported^15,17. At approximately 18 months (range 13 - 21) after the 24 months testing, parents or guardians of the children were contacted and asked to participate in an intervention study in which their children would again receive follow-up medical examination, cognitive testing and cognitive rehabilitation training. Of the 68 children tested at 24 months, 65 were traced and enrolled into this study. Earlier studies with this cohort used children who were 5 years and above at enrollment^15,17 while this study used children who were above five years at 24 months testing, hence the difference in numbers in the above published works and the present study.

Written informed consent was obtained from the parents or guardians of study participants and assent from the children. Ethical approval for this study was granted by the Institutional Review Boards for Human Studies at Makerere University Faculty of Medicine, Michigan State University, University of Michagan and the Uganda National Council for Science and Technology.

Cognitive assessments

Children were tested with the KABC-II which is a measure of information processing and cognitive abilities in children aged 3 to 18 years²³. In assessing information processing in children 7 to 16 years, the KABC-II uses 13 subtests which are grouped into four scales: (a) Learning, storing and efficiently retrieving newly learned or previously learned information; (b) Sequential Processing, taking in and holding information and then using it within a few seconds; (c) Simultaneous Processing, perceiving, storing, manipulating and thinking with visual patterns; and (d) Planning, solving novel problems by using reasoning abilities like induction and deduction.

Of the 13 subtests that were administered, those making up the Learning scale were Atlantis, Rebus, Atlantis Delayed and Rebus Delayed. The Sequential Processing scale was made up of Hand Movements, Word Order and Number Recall. The Simultaneous Processing scale was made of Block Counting, Rover, Triangles and Gestalt Closure. Planning was made up of Pattern Reasoning and Story Completion. Only raw scores were used in the analyses.

After reaching a consensus with the psychological testers and the first author, seven items from Gestalt Closure subtest were removed because they were deemed not appropriate. These seven items represented objects not common in this setting. No other modifications were made to the battery. Table 1 provides a description of the subtests and the modifications that were made to the Gestalt Closure subtest. Prior to testing, all instructions for the KABC-II were translated into Luganda, the commonly spoken language in Kampala by a research assistant and then back translated to English by another research assistant before testing. Both research assistants had training in testing using the KABC-II. The second author, a native Luganda speaker compared the back translation with the original KABC-II instructions to resolve discrepancies.

The Statistical Package for Social Sciences (SPSS) 16.0 was used for analysis. Box plots for all test scores from the KABC-II were produced and all outlying scores excluded from analyses. A principal components analysis with Varimax rotation on the 13 administered subtests was performed to determine the construct validity of the KABC-II. This would help determine whether test performance by Ugandan children on the KABC-II is similar to the American children on which the test was formed. This will in turn answer the question; does assessment of information processing in Ugandan brain injured children using the KABC-II result in the four information processing scales? Factors with eigenvalues greater than 1.00 were retained.

The majority of the children were male (61.5% vs. 38.5%) with a mean age of 9.90 years (SD = 2.55). Fifty two (80%) children had experienced at least one malaria episode in the previous 18 months since their last follow up visit with 5 (7.7%) reporting at least one episode of convulsions. Table 2 gives detailed demographic and health characteristics of the children.

A principal component factor analysis with Varimax solution resulted in five factors having eigenvalues greater than 1.00. Factor loadings of 0.4 and greater were taken as significant. The first factor was composed of three of the four Learning tests (Rebus, Rebus Delayed and Atlantis). The second factor was composed of three of the four Simultaneous Processing tests (Gestalt Closure, Triangles and Block Counting), one Sequential Processing test (Word Order) and one Planning test (Story Completion). All three Sequential Processing tests (Hand Movements, Word Order and Number Recall) loaded on the third factor and so did one Planning test (Pattern Reasoning). The fourth factor was composed of all Planning tests (Story Completion and Pattern Reasoning) and these loadings where much higher than what these individual subtests had on factors two (Story Completion) and three (Pattern Reasoning). The fifth factor did not have tests measuring specific abilities loading on it with Number Recall (Sequential Processing) and Atlantis Delayed (Learning). The above factors accounted for 74% of the variance (see Table 3).

Even after excluding seven items from Gestalt Closure subtest, this subtest loaded with other Simultaneous Processing tests on the same factor and had the highest loading than any other subtest.

Discussion

This study was carried out to determine the construct validity of the KABC-II in Ugandan children with a history of CM. After subjecting thirteen subtests intended to measure Learning, Simultaneous Processing, Sequential Processing and Planning to factor analysis, four factors emerged with subtests measuring these abilities loading on the same factor. These findings suggest that the KABC-II subtests designed to measure Learning, Simultaneous Processing, Sequential Processing and Planning truly measure these abilities in Ugandan children.

We are not aware of published validation studies of the KABC-II in Africa. However validation studies of the original KABC in at risk children showed it to be sensitive to parental education and quality of home environment and it retained its construct validity in these settings^16,19,21. Similarly in our study the KABC-II retained the factor structure as intended by its developers with subtests measuring similar abilities loading on the same factor. The KABC-II managed to retain this construct validity in a group of children whose cognition has been compromised by cerebral malaria^15,17. This goes on to show that the KABC-II can be a valid tool for assessing cognitive function in at risk children in Sub-Saharan Africa.

It has been argued that use of Western tests in other cultures is not appropriate because test items may not mean the same thing in different cultures and there is no universal agreement on the merit of responses²⁶. In solving Western test problems, people in different cultures may use different skills to arrive at solutions suggesting that the tests may measure different skills in different cultural settings²⁷. In light of these limitations of using Western tests in other cultures, this study used the raw scores instead of the scaled scores which are based on Western norms. Our study shows that with standardized test administration involving translation and back translation of instructions and relevant culturally sensitive modifications, these hurdles may to some degree be overcome.

Whereas most subtests measuring similar abilities loaded on the same factor, some subtests loaded on two factors for example Atlantis, Word Order, Number Recall, Pattern Reasoning and Story Completion. This cannot be readily explained but it could be attributed to these subtests measuring abilities found in more than one factor. For example Story Completion is grouped under Planning when assessing children 7 to 18 years but is grouped under Simultaneous Processing for children aged 6 years²³. Similarly, Atlantis Delayed and Number Recall which were the only factors loading on the fifth factor are both measures of memory.

The current study had a number of limitations. It is recommended that factor analysis studies have a sample size of four times the variables under study²⁸. With the 13 variables studied here, our sample size of 65 meets the minimum requirement necessitating caution when interpreting these results. This study can at best be seen as a preliminary study into the validation of the KABC-II as it only focused on the construct validity. Further work is needed to examine the reliability and other forms of validity (for example concurrent and predictive validity) that were not examined.

This study was not able to determine the sensitivity or specificity of the KABC-II in diagnosing cognitive impairment in children as no control children were tested from whom age appropriate norms could be derived. This lack of age appropriate norms in this region demonstrates the need for validated tests from which norms can be developed. Though some studies have modified some test items that may be inappropriate in the culture for which the test were not formed¹⁶, the present study omitted seven test items from the Gestalt Closure and still got high factor loadings for this particular subtest. This implies that the omission of these culturally inappropriate items did not affect this subtest negatively.

To conclude, we believe that using children with a history of CM did not invalidate the findings but instead gave strength to the results. Earlier studies with these same children showed them to have impaired cognition compared to healthy community controls^15,17. The KABC-II retained its factor structure in these children whose cognition has been compromised indicating its potential to assess cognition in Ugandan children who are at risk of cognitive deficits. This is in agreement with the KABC-II's purpose of assessing cognition in children with brain dysfunction or damage²³.

The present study shows that the KABC-II, with some modifications, can be used in the cognitive assessment of children in Uganda. Its construct validity in this Ugandan sample was not affected by back translating instructions from English to the local language and modification by omitting test items that were not appropriate in this culture. We recommend more studies looking into the KABC-II's psychometric properties as this study only examined the construct validity.

Acknowledgement

We would like thank the children who participated, their parents/guardians and the research team who made this study possible. This work was supported in part by NIH grants R21 TW006794 (Fogarty International Center) and 5R01NS055349 (National Institute of Neurological Disorders and Stroke) to Chandy C. John, a University of Michigan Global

References

1. Olness K. Effects on brain development leading to cognitive impairment: a worldwide epidemic. J Dev Behav Pediatr 2003;24(2):120-130.
2. Walker SP, Wachs TD, Gardner JM, et al. Child development: risk factors for adverse outcomes in developing countries. Lancet 2007;369(9556):145-157.
3. Grantham-McGregor S, Cheung YB, Cueto S, Glewwe P, Richter L, Strupp B. Developmental potential in the first 5 years for children in developing countries. Lancet 2007;369(9555):60-70.
4. Boivin MJ, Giordani B, Ndanga K, Maky MM, Manzeki KM, Ngunu N. Economic advantage and the cognitive ability of rural children in Zaire. J Psychol 1996;130(1):95-107.
5. Boivin MJ. Effects of early cerebral malaria on cognitive ability in Senegalese children. J Dev Behav Pediatr 2002;23(5):353-364.
6. Carter JA, Ross AJ, Neville BG, et al. Developmental impairments following severe falciparum malaria in children. Trop Med Int Health 2005;10(1):3-10.
7. Drotar D, Olness K, Wiznitzer M, et al. Neurodevelopmental outcomes of Ugandan infants with human immunodeficiency virus type 1 infection. Pediatrics 1997;100(1):E5.
8. Holding PA, Stevenson J, Peshu N, Marsh K. Cognitive sequelae of severe malaria with impaired consciousness. Trans R Soc Trop Med Hyg 1999;93(5):529-534.
9. Jukes MC, Nokes CA, Alcock KJ, et al. Heavy schistosomiasis associated with poor short-term memory and slower reaction times in Tanzanian schoolchildren. Trop Med Int Health 2002;7(2):104-117.
10. Kihara M, Carter JA, Newton CR. The effect of Plasmodium falciparum on cognition: a systematic review. Trop Med Int Health 2006;11(4):386-397.
11. Richter LM, Grieve KW. Home Environment and Cognitive Development of Black Infants in Impoverished South African Families. Infant Mental Health Journal 1991;12:89-102.
12. Holding PA, Kitsao-Wekulo PK. Describing the burden of malaria on child development: what should we be measuring and how should we be measuring it? Am J Trop Med Hyg 2004;71(2 Suppl):71-79.
13. Bangirana P, Idro R, John CC, Boivin MJ. Rehabilitation for cognitive impairments after cerebral malaria in African children: strategies and limitations. Trop Med Int Health 2006;11(9):1341-1349.
14. Kaufman AS, Kaufman NL. Kaufman Assessment Battery for Children: Administration and Scoring Manual. Circle Pines, MN: American Guidance Service; 1983.
15. Boivin MJ, Bangirana P, Byarugaba J, et al. Cognitive impairment after cerebral malaria in children: a prospective study. Pediatrics 2007;119(2):e360-366.
16. Holding PA, Taylor HG, Kazungu SD, et al. Assessing cognitive outcomes in a rural African population: development of a neuropsychological battery in Kilifi District, Kenya. J Int Neuropsychol Soc 2004;10(2):246-260.
17. John CC, Bangirana P, Byarugaba J, et al. Cerebral malaria in children is associated with long-term cognitive impairment. Pediatrics 2008;122(1):e92-99.
18. John CC, Panoskaltsis-Mortari A, Opoka RO, et al. Cerebrospinal fluid cytokine levels and cognitive impairment in cerebral malaria. Am J Trop Med Hyg 2008;78(2):198-205.
19. Boivin MJ, Chounramany C, Giordani B, et al. Validating a cognitive ability testing protocol with Lao children for community development applications. Neuropsychology 1996;10(4):588-599.
20. Boivin MJ, Giordani B, Bornefeld B. Use of the Tactual Performance Test for cognitive ability testing with African children. Neuropsychology 1995;9(3):409-417.
21. Giordani B, Boivin MJ, Bikayi O, Nseyila DND, Lauer RE. Use of the K-ABC with Children in Zaire, Africa: An Evaluation of the Sequential-Simultaneous Processing Distinction within an Intercultural Context. International Journal of Disability, Development, and Education 1996;43(1):5-24.
22. Ochieng CO. Meta-Analysis of the Validation Studies of the Kaufman Assessment Battery for Children. International Journal of Testing 2003;3:77-93.
23. Kaufman AS, Kaufman NL. Kaufman Assessment Battery for Children Manual. 2 ed. Circle Pines, MN: American Guidance Service; 2004.
24. Caldwell BM, Bradley RH. Home Inventory Administration Manual (3 rd ed.). Little Rock, AR: University of Arkansas; 2001.
25. Kuczmarski RJ, Ogden CL, Guo SS, et al. 2000 CDC Growth Charts for the United States: methods and development. Vital Health Stat 11 2002(246):1-190.
26. Greenfield PM. You Can't Take It with You: Why Ability Assessments Don't Cross Cultures. American Psychologist 1997;52(10):1115-1124.
27. Sternberg RJ. Culture and intelligence. Am Psychol 2004;59(5):325-338.
28. Hammon S. Introduction to Multivariate Data Analysis. In: Breakwell GM, Hammond S, Fife-Schaw C, editors. Research Methods in Psychology. 2 nd ed. London: Sage Publications.; 2000. p. 372-396.

Copyright © 2009 - Makerere Medical School, Uganda

The following images related to this document are available:

Photo images