Determining if reading problems are unexpected is hard: Concerns about the proposed listening comprehension-reading comprehension discrepancy index

At our center, we consume, produce, use, and share research findings. We have ongoing and evolving discussions about research within our center. These discussions also extend to our various stakeholders. A recent article in the Annals of Dyslexia by Tim Odegard, Ph.D.; Emily Farris, Ph.D.; and Julie Washington, Ph.D., contributes to the conversation about the conceptualization of dyslexia.

Background motivating the study

Theoretical models about the identification of dyslexia are multifaceted. Educational policies outlining identification procedures also involve many components. The ability to identify an individual with dyslexia can be challenging. There is no single test or point of data that by itself can be used to definitively confirm an individual has dyslexia.

Closely related to identification issues are discussions of prevalence. Knowledge of how common, or prevalent, a condition is can impact decisions. Prevalence estimates influence funding and resource allocation across many settings. The prevalence estimates for dyslexia range from 5% to 17.5% of the population. Because dyslexia is common, attention and resources are expended on it. Researchers, educators, politicians, and families contribute to understanding and treating individuals with dyslexia.

 

Both prevalence estimates and identification decisions depend on the operational definition of dyslexia. Current definitions of dyslexia begin by describing it. The primary characteristics are difficulties with word reading, decoding, and spelling. Next, constraints are added. They are attempts to create clear boundaries. We want to be able to easily know whether someone who struggles to read well has dyslexia or if the person’s difficulties are due to other reasons. Some of these constraints lead to a search for what caused the reading problems.

Other constraints are set up to show that the individual’s difficulties are isolated to reading. Currently, reading difficulties are described as being unexpected in individuals with dyslexia. The person is described as having reading problems despite receiving effective classroom instruction. They have reading problems despite having adequate hearing and visual acuity. Lastly, their level of performance in reading is unexpected because it differs from their performance in other areas.

Researchers and practitioners continue to discuss the concept of unexpectedness. Often a search for discrepancies ensues. The search is to show that a person’s reading struggles are unexpected. Many of these efforts have been criticized. For example, there are numerous concerns about using IQ scores to demonstrate a discrepancy between a person’s achievement in reading and their intellectual ability (e.g., Siegel, 1989; 1999; Stanovich, 1991; 1999). A substantial concern is that the IQ score represents past learning, not the potential to learn.

Recently, a new way to determine whether reading deficits are unexpected was proposed. It compares an individual’s listening comprehension and reading comprehension abilities. It is the LC-RC discrepancy index. Here LC refers to listening comprehension, and RC refers to reading comprehension. Individuals whose LC is much higher than their RC are proposed to be unexpectedly poor readers. The LC-RC discrepancy index is part of the constellation model of dyslexia (Wagner et al., 2019; 2020). That model emphasizes the need to measure multiple indicators. It also considers contextual factors when determining if an individual has dyslexia. The authors emphasized that the LC-RC discrepancy index should not be used by itself to identify individuals with dyslexia. However, they proposed it could be used to estimate the prevalence of dyslexia.

Concerns about using IQ to measure discrepancy apply to using listening comprehension, too. The listening comprehension scores represent a student’s acquired vocabulary and background knowledge. That knowledge is based on the student’s prior experiences and educational opportunities. The term social determinants can describe factors that comprise a person’s experiences. The World Health Organization refers to social determinants as “the conditions in which people are born, grow, live, work and age” (World Health Organization, 2008). Given our educational focus, we would consider “grow, live, and work” to include conditions where people learn.

The social determinants people experience can heighten their risk for difficulties. They can make those individuals more vulnerable to problems. Further, community resources vary substantially within our country. Many of these resources impact language and early literacy development. They can promote or inhibit growth in vocabulary and background knowledge. If a person's social determinants are less favorable, they may have poor performance on skill-based measures. Those low scores may lead others to underestimate the person's ability and potential.

The LC-RC discrepancy index is likely to be higher in individuals whose social determinants promote the exposure to, knowledge of, and practice using General American English (GAE). Many tests designed to access LC and RC skills use GAE in their items and scoring keys. So, individuals with greater exposure to GAE will likely have higher scores. In contrast, LC-RC discrepancy scores would be smaller for individuals who have less experience with GAE. Some individuals disproportionately experience vulnerability factors that decrease their experiences with GAE. Even if a person has a low RC score, if their LC-RC discrepancy index is small, they would be seen as a person whose reading difficulties are expected. This means that they would not be considered to be a person with dyslexia.

Study overview and hypotheses

We hypothesized vulnerability factors impacting oral and written language development would predict the LC-RC discrepancy index. Specifically, we focused on students who are relatively poor and/or members of minority groups. They are hypothesized to be less likely to have a positive LC-RC discrepancy index. In other words, any potential difficulties they experience in reading are less likely to be seen as unexpected. Consequently, these individuals would be less likely to be identified as individuals with dyslexia.

We used empirical data from public school children to investigate our concerns. We acquired universal screening data from a large sample of children in a single state. The final sample had 4,078 Caucasian or African American students from 117 schools. Poverty was represented by whether or not a student qualified to receive free or reduced-price lunch (FRPL) at school.

The students completed a universal screening of their reading and language skills. This universal screening was standard practice in that state. In our sample, students’ scores across subtests reflected a wide range of performance for both LC and RC.

We calculated the LC-RC discrepancy index for each child. We subtracted RC scores from LC scores, just like Wagner and colleagues did. This LC-RC discrepancy index is the outcome or dependent variable in our statistical analyses. We looked to see if the characteristics of the students (i.e., ethnicity and FRPL status) predicted their LC-RC discrepancy index. Other methodological and analytic details are in our journal article. Our decisions were also informed by statistical debates about discrepancy measures (e.g., Lord, 1967; Trafimow, 2015).

Key Findings

Characteristics of the students (i.e., their ethnicity and FRPL status) predicted their LC-RC discrepancy index. Figure 1 illustrates that the findings support our hypotheses. Students who qualified for FRPL, and especially students who were African American, tended to have less extreme LC-RC discrepancy index scores.

The effect sizes were small to medium. This means that the pattern of findings is reliable. But the actual differences in LC-RC discrepancy index scores will not be extreme. The small effect size can be seen in Figure 1 because the different colored shapes exhibit a lot of overlap. It also helps to look at the students’ LC and RC scores to understand how the groups of students differ from each other.

Figure 2 is an alternate visualization of data from our published article. The colored shapes representing each group of students cover a similar range of scores along the vertical y-axis. Yet the average, or mean, represented by the black dot with standard error bars, varies somewhat across the groups. In particular, the average tends to be lower for poor African American students’ LC scores in light blue relative to the other groups. In contrast, the differences between the groups of students are not as large for their RC scores.

Figure 1. Plot of the listening comprehension-reading comprehension (LC-RC) Discrepancy index by student ethnicity and free-reduced pric lunch (FRPL) status		Figure 2. Plot of listening comprehension (LC) and reading comprehension (RC) scores by student ethnicity and free-reduced price lunch (FRPL) status

 

Take-away messages

We agree with prior studies that the LC-RC discrepancy index can show that the prevalence of dyslexia varies with severity. But our results highlight some challenges involved in using it. We need to carefully consider social determinants of language development. These factors may also impact performance on measures of language skills.

We also caution readers that there is not a simple solution. Making a more sensitive test that accounts for cultural and language differences probably will not solve the problem. Moreover, it is unlikely that we can simply develop a more complex statistical model to overcome these biases. Instead, there are harder questions that need to be asked. These questions involve how dyslexia and other learning disabilities are defined.

Specifically, we should continue to discuss unexpectedness. Is unexpectedness a useful concept? Does it make a substantial difference to our treatment plans if a poor reader’s struggles are unexpected? Must a child’s reading struggles be unexpected for them to receive the support and protections afforded to individuals with dyslexia? What protocols do we have to support any student who struggles to read? How can we account for the influence of social determinants on reading ability in our identification models? These are questions in our hearts and minds. They are likely being pondered by our stakeholders too.

References

• Lord, F. M. (1967). A paradox in the interpretation of group comparisons. Psychological Bulletin, 68(5), 304–305. https://doi.org/10.1037/h0025105
• *Odegard, T. N., Farris, E. A., & Washington, J. A. (2022). Exploring boundary conditions of the listening comprehension-reading comprehension discrepancy index. Annals of Dyslexia. https://doi.org/10.1007/s11881-021-00250-0. This is the article being summarized.
• Siegel, L. S. (1989). IQ is irrelevant to the definition of learning disabilities. Journal of Learning Disabilities, 22(8), 469–478. https://doi.org/10.1177/002221948902200803
• Siegel, L. S. (1999). Issues in the definition and diagnosis of learning disabilities: A perspective on Guckenberger v. Boston University. Journal of Learning Disabilities, 32(4), 304–319. https://doi.org/10.1177/002221949903299405
• Stanovich, K. E. (1991). Conceptual and empirical problems with discrepancy definitions of reading disability. Learning Disability Quarterly, 14(4), 269–280. https://doi.org/10.2307/1510663
• Stanovich, K. E. (1999). The sociopsychometrics of learning disabilities. Journal of Learning Disabilities, 32(4), 350–361. https://doi.org/10.1177/002221949903200408
• Trafimow, D. (2015). A defense against the alleged unreliability of difference scores. Cogent Mathematics, 2(1), 1064626. https://doi.org/10.1080/23311835.2015.1064626
• Wagner, R. K., Edwards, A. A., Malkowski, A., Schatschneider, C., Joyner, R. E., Wood, S., & Zirps, F. A. (2019). Combining old and new for better understanding and predicting dyslexia. Models for Innovation: Advancing Approaches to Higher-Risk and Higher-Impact Learning Disabilities Science. New Directions for Child and Adolescent Development, 165, 1–13. https://doi.org/10.1002/cad.20289
• Wagner, R. K., Zirps, F. A., Edwards, A. A., Wood, S. G., Joyner, R. E., Becker, B. J., Liu, G., & Beal, B. (2020). The prevalence of dyslexia: A new approach to its estimation. Journal of Learning Disabilities, 53(5), 354–365. https://doi.org/10.1177/0022219420920377
• World Health Organization. (2008). World Health Organization, Commission on Social Determinants of Health. Closing the gap in a generation: Health equity through action on the social determinants of health. CSDH final report. World Health Organization.

---

 

 

Emily Farris, Ph.D.,
Assistant Director for Educational
Services and Research Initiatives