EVOPEDIA

← all systems · see it in The System

Standardized test

A standardized test is a test that is administered and scored in a consistent or standard manner.

Properties published scienceour proposal

Sourced fields, and our own readings.

born atearliest evidence in Han China (encyclopedia); the psychometric form from Galton's work (late 1880s) and Binet and Simon's scale (commissioned 1904), mass group testing from 1917
born precisioncenturies (first evidence); years (the psychometric form)
boundaryInside: the standard form and conditions, the items and item bank, the taker's responses, the scoring and scaling rules (CTT, IRT, equating), the norms or criterion and the score report. Outside: the takers as persons (Human), their schooling and families, the institutions that set stakes and use scores (states, schools, universities, employers, the army), and the coaching market.
nested levelA kind of examination (rung 11), built from items, takers, scoring rules and norms; part of school systems, admissions and licensing; carried by psychometrics as its science.
scalesocial

Measures (26) published science

PISA cycle (characteristic clock)triennial (reading 2000, mathematics 2003, science 2006 as main domains, repeating) years
re-norming interval (WISC editions)1949, 1974, 1991, 2003, 2014 (11-25 years apart)
Flynn effect, full-scale IQ0.28 (fluid 0.41, spatial 0.30, crystallized 0.21); 271 samples, almost 4 million people, 31 countries, 1909-2013 IQ points per year
time-of-day effect on test scores-0.9% of an SD per hour later (95% CI 0.7-1.0); +1.7% after a 20-30 minute break
coaching effect on SAT I (median estimates)verbal 8, math 18 (Becker's meta-analysis of 48 studies: 9 and 19) SAT points
coaching yieldabout one extra math question per eight hours of effort; one verbal question per 25-30 hours questions per hour
scorer agreement (human scoring)60-85 percent
direct cost of US state tests (2012)$27 per student; $669 million across 45 states US dollars
SAT-intelligence correlation0.5-0.9 (up to 0.8 with fluid reasoning and g in Frey and Detterman)
SAT's added predictive power over HSGPAabout 15 percent
first-year GPA predicted by HSGPA and SAT (34 colleges)average R-squared .226
college average SAT/ACT vs US News rankingvery nearly 0.9
overprediction of Black students' first-year GPA.11-.24 GPA points; reduced 54-72% by omitted variables (N = 499,622, 142 institutions)
score inflation (math, state vs NAEP)state +0.088 vs NAEP +0.048 SD per year
NCLB effect on 4th-grade math (NAEP)0.22 by 2007 (no reading effect) SD (effect size)
income achievement gap growthabout 75 percent larger for the 2001 cohort than for the early-1940s cohort
Black-White gap narrowing (cohorts mid-1950s to mid-1970s, NAEP)about 0.5 SD
WWI army testingnearly two million men examined; about 5% scored very poorly and were retested individually people
test anxiety prevalence10-40 percent of students
colleges dropping SAT/ACT requirements after COVID (2020)more than 1,600 (60%) US colleges
coaching uptake (1995-96 SAT takers)nearly 12 percent
standard error of measurement (formula)SD of observed scores times the square root of (1 - reliability) score points
median elementary school size per grade (accountability sample)69 children per grade
Flynn effect turning point (Norway)rise, turning point and decline within cohorts born 1962-1991 birth cohorts
PISA mode effect (computer minus paper, randomized field trial)maths about -0.10 to -0.20; reading -0.15 to -0.20; science Germany -0.25, Sweden -0.07 SD
pollution effect on an exam score (Israel, Bagrut)1 SD more PM2.5 (AQI): -0.93 points, 3.9% of an SD; +10 units across exams: -109 shekels ($30) monthly salary in 2010 points; shekels

Open contentions (11) open contention

Where sources or readings disagree. Both stay on the map.

Cross-scale patterns seen here (15) our proposal

Shapes that recur when the same kind of problem is solved at different scales, and how each shows in Standardized test. Each links to every other system that shows it.

Valence: what it trades, and with whom (13) published science

Each exchange with a neighbour, read from both sides: who gains, who pays, and what carries it.

Partners and parts published science

Built from / builds (5)

Sits inside / holds (4)

expressed by (9)

floor filled by (12)

represents (1)

Kinds (12)

wired (1)

Notes published science

Layers

The ways it takes in its world, where they meet, and how it acts back, each from L1 (where it meets the world) up.

standardized test: from a population of test takers to scores read the same way for all, and the decisions they open (reader's view)

  1. L1: The tested world: a population of test takers (students of an age, applicants, draftees) and the domain of knowledge or ability to be measured
  2. L2: The standard form and conditions: the same questions, instructions, timing and scoring rules for all; registration, routing to a form (Army Alpha or Beta), accommodations
  3. L3: The items: multiple choice, true-false, essays, performances; written, tried out and calibrated (IRT) into a bank
  4. L4: The response under standard conditions: where the construct meets construct-irrelevant variance (fatigue, anxiety, air)
  5. L5: Scoring and scaling: machine and rubric scoring; true score plus error; IRT ability estimates; equating
  6. L6: Meaning: norms or criterion; reliability and validity of the interpretation for a use
  7. L7: Use: admission, licensing, accountability, comparison

the boardroom (shared floors)

  1. L5: Scoring and scaling: machine scoring of multiple choice, rubric scoring with two or more scorers (agreement 60-85 percent); the observed score read as true score plus error (classical test theory) or as an ability estimate on a scale shared with the items (IRT); equating of forms so scores are interchangeable
  2. L6: Meaning: the score read against norms (a standardization sample) or a criterion (a cut score); validity as evidence and theory supporting an interpretation for a use (five sources of evidence); reliability a prerequisite, not a guarantee; group means more trustworthy than single scores
  3. L7: Use acting on the world: admission, placement, licensing, diagnosis, school accountability, national and international comparison

way in: Delivery mode (paper or screen)

  1. L1: The medium of the test itself: paper booklets or computer delivery (PISA moved most countries to computer in 2015)
  2. L2: The same questions presented on paper or on screen, pupils randomly assigned within schools (field trial)
  3. L3: Some questions function differently across modes; with those removed the estimates fall and most lose significance
  4. L4: Scores shifted: maths -0.10 to -0.20 SD and reading -0.15 to -0.20 SD on computer; science -0.25 Germany, -0.07 Sweden

way in: Item tryouts and item statistics (the item bank)

  1. L1: Draft items written to cover a domain
  2. L2: Items tried out and adapted; for PISA translated and adapted from two source versions, English and French
  3. L3: Item statistics under IRT: difficulty, discrimination, guessing; differential item functioning screens (the R-SAT used only the hardest items)
  4. L4: The calibrated item bank: items and abilities on one scale, from which forms and adaptive tests are assembled

way in: Low-stakes audit tests (NAEP, audit items)

  1. L1: A second, low-stakes measurement of the same students or states (NAEP)
  2. L2: Given without the high-stakes test's incentives
  3. L3: Trends on the audit and the high-stakes test compared; the comparison carries a content confound (different exams sample different parts of the domain)
  4. L4: Audit items built into the high-stakes test itself (self-monitoring assessments)
  5. L5: Gaps between the two read as inflation

way in: Test preparation and coaching

  1. L1: A commercial test-prep market, its claims (a guaranteed 100 points)
  2. L2: About 12 percent of 1995-96 SAT takers attended outside coaching programs
  3. L3: Practice on the test's item types; uncoached takers also gain on retesting
  4. L4: Small score change: about 8 verbal and 18 math points (median estimates)

way in: Test takers arriving (the tested population)

  1. L1: The population a test is given to: all students of a certain age, all adults who wish to acquire a licence for a particular job
  2. L2: Registration and routing to the right form: the Army sorted the literate to the Alpha and the rest to the Beta, by asking whether a man could read a newspaper and write a letter home; accommodations for disability that keep the point of the test
  3. L3: Seated under the same conditions: same test, same time, same circumstances
  4. L4: Sitting the test (paper, computer, computer-adaptive, practical); in sample surveys (PISA, NAEP) each student sits only part of the items and results are population estimates (plausible values)

way in: The body and conditions on the test day (construct-irrelevant variance)

  1. L1: The air, the clock and the body on the day: PM2.5 during Israeli matriculation exams; the hour of the test in Danish schools
  2. L2: Felt in the taker: 10-40 percent of students experience test anxiety
  3. L3: Associated with lower scores: all three dimensions of test anxiety negatively associated with high-stakes reading scores (correlational; the authors cite cognitive load theory's working-memory account as one explanation)
  4. L4: Performance shifted: -0.9% of an SD per hour later in the day, +1.7% after a 20-30 minute break

way in: The norming (standardization) sample

  1. L1: The population whose performance moves over time: IQ gains of about 0.28 points a year across 1909-2013 (Flynn effect); slowed or reversed in several countries (Norwegian conscripts born 1962-1991)
  2. L2: A standardization sample drawn from it takes the test under standard conditions
  3. L3: Its score distribution
  4. L4: The norms: IQ 100 defined as the sample's mean; percentiles and ranks
  5. L5: Norms age as the population moves; WISC re-normed 1974, 1991, 2003, 2014; stale norms meet the stakes: all 50 states reported above the national average on normed tests (Cannell 1988)

way in: What rides on the score (stakes and incentives)

  1. L1: Laws and institutions that attach rewards and sanctions: ESEA 1965, NCLB 2001, performance pay, admissions
  2. L2: Incentives reaching schools, teachers and takers
  3. L3: Responses: teaching to the test, excluding low scorers, cheating
  4. L4: Scores rising on the high-stakes test
  5. L5: Score inflation in the trend: state math +0.088 SD a year against NAEP +0.048

way in: What the taker knows and can do, drawn out by the items (the construct)

  1. L1: The taker's knowledge, skills and abilities: the construct the scores are meant to indicate
  2. L2: The item as stimulus: the same questions for everyone, read or heard
  3. L3: The response process: reasoning and retrieval; SAT scores correlate 0.5-0.9 with measures of intelligence (g)
  4. L4: The marked response; under the 3PL model a lower asymptote gives low-ability takers a chance of a correct answer

way out: Decisions on individuals (admission, placement, licence)

  1. L1: People sorted: some commanding officers balanced units by score, others ignored scores
  2. L6: Score reports and ranks; colleges' average SAT/ACT tracks their US News ranking (r near 0.9)
  3. L7: The decision: admission, a licence, an army assignment

way out: National and international comparison (NAEP, PISA, TIMSS, PIRLS)

  1. L1: Education policy formed on PISA data in many countries
  2. L6: Country reports and secondary analyses (654 peer-reviewed PISA articles, 2000-2015)
  3. L7: Triennial PISA results; international benchmarks; population estimates from an incomplete design (plausible values), not individual scores like an admission test

way out: School accountability and backwash

  1. L1: Classrooms changed: what is not tested is not taught
  2. L3: Instruction narrowed to what is tested
  3. L6: Ratings and rankings of schools and teachers; school means are imprecise (median elementary school: 69 children per grade), so rewards and sanctions at the extremes fall mostly on small schools
  4. L7: Funds allocated and poorly performing schools closed on test results

way out: Scores as data for research

  1. L1: The research literature and the policy debate
  2. L6: Findings: income gaps 75 percent larger for the 2001 cohort than the early-1940s cohort
  3. L7: Test data used to study gaps, growth and intelligence (SAT as a proxy for g)

Sources (24) published science

Dee & Jacob 2009, The Impact of No Child Left Behind on Student Achievement, NBER Working Paper 15531
Standardized test (Wikipedia, via Wikimedia Enterprise snapshot)
Campbell DT (1976/1979) Assessing the impact of planned social change. Occasional Paper 8, Dartmouth Public Affairs Center (1976); Evaluation and Program Planning 2(1):67-90 (1979), doi:10.1016/0149-7189(79)90048-X. Read in sections (the law and the achievement-test passages) in the open reprint, Journal of MultiDisciplinary Evaluation 7(15), 2011 (own paced fetch; the portal gave the record only) (behind its links)
Koretz DM, Jennings JL, Ng HL, Yu CY, Braslow D, Langi M (2016) Auditing for score inflation using self-monitoring assessments: findings from three pilot studies. Educational Assessment. doi:10.1080/10627197.2016.1236674 (abstract only, paper portal) (behind its links)
Flynn effect (Wikipedia, via Wikimedia Enterprise snapshot 2026-11-01) (behind its links)
Himelfarb I (2019) A primer on standardized testing: history, measurement, classical test theory, item response theory, and equating. Journal of Chiropractic Education. doi:10.7899/jce-18-22 (full text reached through the paper portal via OpenAlex, read in sections) (behind its links)
Amrein-Beardsley A, Azizova ZT, Gibbs NP, et al. (2025) A validation review of the SAT and ACT for college and university admissions decisions. Education Policy Analysis Archives 33. doi:10.14507/epaa.33.8734 (full text reached through the paper portal via OpenAlex, read in sections) (behind its links)
Powers DE, Rock DA (1998) Effects of coaching on SAT I: Reasoning scores. ETS Research Report Series 1998(1). doi:10.1002/j.2333-8504.1998.tb01784.x (full text reached through the paper portal via OpenAlex, read in sections) (behind its links)
Hopfenbeck TN, Lenkeit J, El Masri Y, Cantrell K, Ryan J, Baird J-A (2017) Lessons learned from PISA: a systematic review of peer-reviewed articles on the Programme for International Student Assessment. Scandinavian Journal of Educational Research. doi:10.1080/00313831.2016.1258726 (full text reached through the paper portal via OpenAlex, read in sections) (behind its links)
Pietschnig J, Voracek M (2015) One century of global IQ gains: a formal meta-analysis of the Flynn effect (1909-2013). Perspectives on Psychological Science. doi:10.1177/1745691615577701 (abstract only, paper portal) (behind its links)
Sievertsen HH, Gino F, Piovesan M (2016) Cognitive fatigue influences students' performance on standardized tests. PNAS 113(10):2621-2624. doi:10.1073/pnas.1516947113 (PMC4790980) (behind its links)
Frey MC (2019) What we know, are still getting wrong, and have yet to learn about the relationships among the SAT, intelligence and achievement. Journal of Intelligence. doi:10.3390/jintelligence7040026 (PMC6963451; full text reached through the paper portal, read in sections) (behind its links)
Demeke S, Sackett PR, Kuncel NR, Lee SD (2026) An omitted variables approach to disentangling differential prediction in college admissions. Educational Measurement: Issues and Practice. doi:10.1111/emip.70048 (abstract only, paper portal) (behind its links)
Jacob BA (2007) Test-based accountability and student achievement: an investigation of differential performance on NAEP and state assessments. NBER Working Paper 12817. doi:10.3386/w12817 (full text reached through the paper portal via OpenAlex, read in sections) (behind its links)
Reardon SF (2018) The widening academic achievement gap between the rich and the poor. Routledge eBooks, doi:10.4324/9780429499821-33 (full text reached through the paper portal via OpenAlex, read in sections) (behind its links)
Pintner R (1921) Review of Psychological Examining in the United States Army (Yerkes RM, ed., Memoirs of the National Academy of Sciences 15). Journal of Educational Psychology 12. doi:10.1037/h0068671 (the whole short review read, paper portal via OpenAlex) (behind its links)
Kane TJ, Staiger DO (2002) The promise and pitfalls of using imprecise school accountability measures. Journal of Economic Perspectives. doi:10.1257/089533002320950993 (abstract only, paper portal) (behind its links)
Bratsberg B, Rogeberg O (2018) Flynn effect and its reversal are both environmentally caused. PNAS. doi:10.1073/pnas.1718793115 (abstract only, paper portal; PMC full text not open to programs) (behind its links)
Jerrim J, Micklewright J, Heine J-H, Saelzer C, McKeown C (2018) PISA 2015: how big is the 'mode effect' and what has been done about it? Oxford Review of Education. doi:10.1080/03054985.2018.1430025 (full text reached through the paper portal, read in sections: abstract, design, results, conclusion) (behind its links)
Ebenstein A, Lavy V, Roth S (2016) The long-run economic consequences of high-stakes examinations: evidence from transitory variation in pollution. American Economic Journal: Applied Economics 8(4):36-65. doi:10.1257/app.20150213 (published version read in sections, introduction and results summary, from the LSE Research Online copy by own paced fetch; the portal gave the abstract) (behind its links)
Test validity (Wikipedia, via Wikimedia Enterprise snapshot; abstract and history sections read) (behind its links)
Warne RT, Burton JZ, Gibbons A, Melendez DA (2019) Stephen Jay Gould's analysis of the Army Beta test in The Mismeasure of Man: distortions and misconceptions regarding a pioneering mental test. Journal of Intelligence. doi:10.3390/jintelligence7010006 (full text reached through the paper portal, read in sections) (behind its links)
Wood SG, Hart SA, Little CW, Phillips BM (2016) Test anxiety and a high-stakes standardized reading comprehension test: a behavioral genetics perspective. Merrill-Palmer Quarterly 62(3):233-251. doi:10.13110/merrpalmquar1982.62.3.0233 (PMC5487000; full text reached through the paper portal, read in sections) (behind its links)
Teaching to the test (Wikipedia, via Wikimedia Enterprise snapshot; abstract and reference list read) (behind its links)