Cultural Bias in High-Stakes Educational Testing
Critical Race Theory and racial bias in public education have become particularly hot political topics in recent months. Often, such terms are thrown about vaguely, with little real understanding of their prevalence or their application. Generalized opinions on both sides of the metaphorical debate stage are detrimental to our students, as they fail to address specific, solvable problems within the school system in the area of racial or cultural bias. This paper explores three studies related to bias in the context of high-stakes standardized testing, and draws themes from these studies to encourage further discussion.
English Language Learners and High-Stakes Testing: A Texas Study
When exploring the implications of racial and cultural bias in education, an excellent sample population is Texas, where over 50% of public school students are Hispanic, and nearly 20% are English Language Learners (Bach, 2020, p. 18). Dr. Amy J. Bach, Assistant Professor of Literacy at the University of Texas El Paso, conducted a 22-month study of high school students in El Paso to explore the gaps and problems within high-stakes assessments for multilingual students. The study consisted of classroom observations, interviews, and data collection which was then compared and cross-examined in order to identify themes and similarities.
One of the findings of the study was the impact of “test-centric instruction” (Bach, 2020, p.25). This type of instruction is motivated by test scores and testing content; students are taught what they will need to know in order to do well on a standardized test. In Texas, the STAAR exam is used not only as a demonstration of student academic performance, but as a requirement for graduation (Bach, 2020, p.22). Because of this, the study found that many teachers placed significant energy and focus on helping their students prepare for the STAAR exam. According to J. Cummins, a student needs at least 5 years of exposure to a new language in order to achieve academic proficiency in that language (2000, referenced in Bach, 2020, p.27). But as Bach observed, “…the State [of Texas] demands the rapid acquisition of English academic language proficiency from newcomer students in order to achieve proficiency on the reading an writing competencies assessed on the STAAR English I and II exams and punishes emergent bilingual high school students if they do not by denying them their diploma” (2020, p.27–28). The solution many teacher found was to instruct students based on the STAAR exam requirements, rather than the skills needed to truly gain proficiency in English.
In focusing on testing rather than learning, teachers participating in this study also tended to base essay assignments on “formulas”, rather than on higher-level thinking or written expression (Bach, 2020, p.26). This approach devalues writing as a tool that is used in many walks of life, and that is necessary for thoughtfully communicating ideas. It creates a disconnect between the mechanics of writing and the student’s individual critical thinking skills. The study found that much English work took place online, where discussion and communal learning were replaced with streamlined, solitary programming (Bach, 2020, p.26). While this approach perhaps helped students to learn English more quickly, it removed the crucial language piece of communication with peers.
This study was very limited, as the participants were from one high school with a uniquely large Hispanic population. However, it did identify patterns in how teachers approach instruction of English Language Learners, and some potential harmful effects of high-stakes testing on English Language Learners. It addressed the need to reexamine these standardized tests and make them more accessible for all students, especially those who are still growing in their English proficiency. It also challenged teachers to rethink their instruction to ensure that students are learning, not just memorizing test-taking strategies or formulas.
The Emotion of Language in Assessments
One aspect of language often overlooked is emotion. For each word we know, we connect some sort of sensory experience or emotion to it, a phenomenon known as emotioncy (Karami, Pishghadam, & Baghaei, 2019). A study in the city of Kerman, Iran, analyzed the connection between participants’ familiarity with an English word and their emotioncy toward that word. The participants were 235 English as a Foreign Language (EFL) students between the ages of 12 and 44, whose native language was Persian. Each participant took a multiple-choice vocabulary test of 20 English words. They then rated their familiarity with the word, and their emotional connection to the word. For example, if the vocabulary word was ‘shrimp’, the student indicated whether they had heard of ‘shrimp’, experienced ‘shrimp’, or perhaps studied ‘shrimp’ (Karami, et. al., 2019). They also indicated whether they had negative, neutral, or positive feelings toward the word (Karami, et. al., 2019).
The study found that there was a definite correlation between the participants’ level of familiarity and emotional connection with a word, and their ability to correctly identify that word (Karami, et. al., 2019). According to the authors, language is not the only factor in determining a person’s emotioncy toward a certain word. Cultural background, gender, social class, and other personal identifiers also play a role in how individuals process language and what type of exposure to language they have (Karami, et. al., 2019). The deeper an individual’s personal experience with a word (or term), the more accurately they are able to interact with that word within an assessment (Karami, et. al., 2019).
Though this study was limited in its participants, it revealed important data which could lead to broader studies regarding emotioncy and standardized testing. One of the major implications of this study is that students’ personal experiences (or lack thereof) with various terms and phrases in a test are likely to impact their performance on that test. If a student only has a general understanding of a certain term, they will likely perform differently than if they have personally experienced that thing (Karami, et. al., 2019). The authors suggest that test-writers be more conscious of the words and scenarios they use, and strive to be more inclusive of a variety of cultures and socioeconomic backgrounds (Karami, et. al., 2019).
Effects of Racial Stereotypes on Test Performance
English Language Learners are not the only students who experience cultural bias in standardized testing. In a qualitative study performed at an urban elementary school in Florida, Martin J. Wasserberg (University of North Carolina-Wilmington) interviewed and observed 4 high-achieving African American students multiple times throughout a school year regarding their attitudes toward test-taking (2017). Wasserberg focused his study around the concept of “stereotype threat”, the fear of an individual that others are judging them based on a stereotype (2017, p.41). According to the study, previous research shows a primary stereotype that affect the academic success of African American students is the false notion that African Americans are less intelligent (Wasserberg, 2017, p.41).
The students interviewed indicated that they believed most of their education centered around test-taking and test preparation (Wasserberg, 2017, p.43). They also verbalized and demonstrated test anxiety due to their teachers’ constant reminders about standardized testing, and regular practice test activities (Wasserberg, 2017). Perhaps most detrimental, the students identified low test scores with their race. The school used for the study had a predominantly African-American population, and one student in particular expressed that White people would most likely assume they were a low performing school because of that (Wasserberg, 2017, p.45). According to Wasserberg, “The participants felt a responsibility to raise the school’s grade, thereby repudiating stereotypes” (2017, p.46).
The particular school observed in this study practiced test-centered instruction, so the results of the interviews and the student perspectives were specific to that type of learning environment. Also, as a qualitative study, the purpose was to explore student perspective, not collect broad data. This study revealed that African American students as young as elementary school feel extra pressure to perform well on standardized tests, so that their schools are not identified as less-than or low-achieving (Wasserberg, 2017). The students in this study did not feel their education was worth much more than a grade on a test. This raises an important challenge to further explore the impact of racial stereotypes on students in the testing environment, as well as to change the approach of certain schools and curriculums to be less test-centered.
Discussion
Two major themes can be drawn from these three unique research studies. The first is that, while test-centered instruction may seem like a fast-track for students to achieve higher scores on standardized tests, it is detrimental to both African American students and English Language Learners. Test-centered instruction misses important steps in the learning process, exchanging higher-level thinking and individual expression for surface-level testing strategies. Because of this, English Language Learners have fewer opportunities to digest and internalize the language in which they are being immersed. Test-centered instruction also places incredible pressure on students, particularly those of color who already experience racial stereotypes. These students may have especially strong feelings of needing to prove their intelligence, and focusing their education solely on passing a high-stakes test only furthers this misconception.
The second theme is that experience plays into student test performance more than we realize. A student is likely to perform differently on a test depending on whether they are familiar with the terms, examples, and scenarios within that test. Small nuances within the test’s wording might impact a student’s score, which in turn may impact their educational trajectory. This places an unfair disadvantage on English Language Learners, as well as those of lower socio-economic status who may not have had exposure to certain cultural experiences deemed ‘normal’ or ‘common’ by test-makers. Authors should consider adding more varied cultural experiences and terminology into high-stakes tests, as well as language accommodations for those who are not native English speakers, in order to obtain a more fair representation of students’ academic achievement without the barrier of language and culture.
References
Bach, A.J. (2020). High-stakes, standardized testing and emergent bilingual students in Texas: a call to action. Texas Journal of Literacy and Education, 8(1), 18–37. Retrieved from https://files.eric.ed.gov/fulltext/EJ1261335.pdf
Cummins, J. (2000). Language, power, and pedagogy: Bilingual children in the crossfire. Multilingual Matters.
Karami, M., Pishghadam, R., & Baghaei, P. (Mar 2019). A probe into EFL learners’ emotioncy as a source of test bias: Insights from differential item functioning analysis. Studies in Educational Evaluation, 60, 170–178. Retrieved from https://doi.org/10.1016/j.stueduc.2019.01.003
Wasserberg, M.J. (2017). High-achieving African American elementary students’ perspectives on standardized testing and stereotypes. The Journal of Negro Education, 86(1), 40–51. Retrieved from https://www.jstor.org/stable/10.7709/jnegroeducation.86.1.0040
