EVOPEDIA

← all systems · see it in The System

Examination

An examination (exam or evaluation), or test, is an educational assessment intended to measure a test-taker's knowledge, skill, aptitude, physical fitness, or classification in many other topics (e.g., beliefs).

Properties published scienceour proposal

Sourced fields, and our own readings.

aliasesExaminations and leaving certificates
born at165 BC (Han recruitment through examinations); written examinations in earnest under the Sui (605 or 607)
born precisiondecades
boundaryInside: the formal occasion of testing and its machinery: the tasks set (questions, items, essays, performances), the candidate answering under set conditions (place, time, invigilation, identity checks), the marking and scoring of the answers (anonymised papers, markers, rubrics, machines), the grade or rank read against a norm or a criterion, and the decision it opens (pass, rank, admission, licence, appointment). Outside: the learning before it (school, university, tutors, the candidate's own memory), the institutions that set the stakes and use the result (state and bureaucracy, universities, professional bodies, employers), and the candidate as a person (walked as Human).
nested levelA practice and institution of human society (rung 11): built from candidates, tasks, examiners and a scoring rule; a part of schools, universities, bureaucracies and licensing bodies (their door or their leaving gate). The family runs from informal oral tests to imperial civil-service examinations, written school exams, standardized tests and professional licensing.
scalesocial

Measures (35) published science

examination cycle (characteristic clock)triennial for the higher-level degrees (decreed 1067; nominal in practice); entry-level examinations every year years
length of one sitting (clock)three sessions (Ming provincial and metropolitan), the second three days after the first and the third three days after that; each could take up to three days and two nights of writing
imperial examination lifetime1,300 years, to 1905
Tang jinshi pass rate1-2 percent; 6,585 jinshi in the dynasty, about 23 a year
Song prefectural candidates per year100,000 (mid-12th century); more than 400,000 (mid-13th century) candidates
Song degree-holders from non-official familiesabout 57 percent (1148 and 1256 lists)
Ming metropolitan graduates from official families14 percent early Ming; about 60 percent late Ming
shengyuan per headone per 300 people (1600); one per thousand (mid-19th century) ratio
Qing jinshi26,747 in 112 examinations, 1644-1905; 238.8 per examination degrees
late Qing pass rate1 percent of two or three million annual applicants
shengyuan to jinshione in three thousand ratio
average age of jinshi candidates (settling time)thirties years of age
Tang Grand Chancellors from the examinationabout 85 percent by the ninth century
entry-level quota stepsmost common 8, 12, 15 and 20 per county or prefecture places
agreement between human scorers60 to 85 percent percent agreement
error variance at reliability 0.800.20 (true-score share 0.80), by the definition that reliability is the true-score share of score variance fraction of score variance
acceptable Cronbach's alpha0.70 to 0.95 (maximum 0.90 recommended) coefficient
time-of-day effect on test scores-0.9% of an SD per hour later (95% CI 0.7-1.0%); a 20-30 min break +1.7% of an SD
pollution effect on exam scores1 SD of PM2.5 AQI (22.81) = -0.65 points (2.8% of an SD); AQI above 101 = -1.95 points (8.2% of an SD) Bagrut points
testing effect size (meta-analyses)d/g 0.50 to 0.61 effect size
students with test anxiety10 to 40 percent
test-anxiety score gapabout 12 percentile points
classrooms with teacher cheating (Chicago)at least 4-5 percent a year (3.4-5.6 percent across specifications) percent of classrooms
Chicago probation thresholdunder 15 percent of students at or above national norms in reading
college students who have cheated50 to 70 percent at least once (US, since the 1960s studies)
Ivy-Plus attendance gap at equal scorestop-1% children more than twice as likely as middle-class children with comparable SAT/ACT scores ratio
stereotype threat effect (girls, MSSS tests)d = -0.22 over 47 effect sizes; signs of publication bias
stereotype threat effect (registered replication)d = -0.006 (N = 822) against the original d = -0.82 (N = 45) effect size
licensing score and care+3.8 consultations, -2.7 inappropriate prescriptions, +6.6 mammography referrals per 1000 patients per SD of score
US state testing spend 2012$669 million; $27 per student (45 states) USD
China legal qualification exam pass rate10 to 15 percent a year (from 2018)
test bank sizeup to four thousand questions items
Yuan provincial quota300 places, 75 per group; 28 Han Chinese from South China among them
NCLB effect on NAEP 4th-grade mathseffect size 0.22 by 2007; no evidence of an effect on reading in grades 4 or 8
standard-setting panel size5-15 subject-matter experts people

Open contentions (12) open contention

Where sources or readings disagree. Both stay on the map.

Cross-scale patterns seen here (18) our proposal

Shapes that recur when the same kind of problem is solved at different scales, and how each shows in Examination. Each links to every other system that shows it.

Valence: what it trades, and with whom (14) published science

Each exchange with a neighbour, read from both sides: who gains, who pays, and what carries it.

Partners and parts published science

Built from / builds (9)

Sits inside / holds (5)

expressed by (16)

floor filled by (16)

represents (1)

Kinds (24)

wired (1)

Notes published science

Layers

The ways it takes in its world, where they meet, and how it acts back, each from L1 (where it meets the world) up.

examination: from what a candidate has learned to a score and the decision it opens (reader's view)

  1. L1: The examination's world: candidates who have prepared (in schools, with tutors and cram schools, for years), the body of knowledge or skill a syllabus sets down, and the offices, places and licences waiting on a decision
  2. L2: The door: registration, eligibility and identity; guards searching for hidden texts; invigilators present to instruct and to prevent cheating
  3. L3: The task: items and papers drawn from the syllabus (multiple choice, completion, essay, oral question, performance); set forms such as the eight-legged essay; tiers of examinations from local to provincial to court
  4. L4: The response under set conditions: the candidate answering in the hall or cell, on paper, computer, by voice or by performing; fatigue, anxiety and the air on the day enter here
  5. L5: Marking and scoring: anonymised and recopied papers, several markers and a determining official, uniform scoring, rubrics, machine scoring, statistical checks; reliability
  6. L6: Meaning: the score or grade read against a norm or a criterion, its reliability and validity; the quota or pass mark that sets the cut; validity is judged for the interpretation and use, not the test; cut scores come from a quota or from a standard-setting panel and a policy decision
  7. L7: The decision acting on the world: passing, ranking, admitting, licensing, appointing; feedback into learning; and backwash on what is taught

the boardroom (shared floors)

  1. L5: Marking and scoring: anonymised and numbered papers recopied three times, two markers and a determining official (imperial China); uniform administration and scoring, rubrics, double scoring, machine scoring; statistical checks for cheating; reliability estimated (alpha 0.70-0.95 acceptable); score linking and equating across forms; differential item functioning screens and accommodations (fairness as a validity issue). Alpha, the usual estimate, is Guttman's lambda-3, a lower bound that many psychometricians deplore relying on alone (Revelle)
  2. L6: What the score means for a use: validity is a property of the interpretation of scores for a proposed use, not of the test, and validation is an argument for and against that interpretation (Standards); threats are construct underrepresentation and construct-irrelevant variance; reliability caps it; read against a norm (rank, curve) or a criterion fixed by a standard-setting panel and adopted by a policy body, or against a quota; bent by the stakes riding on it
  3. L7: The decision acting on the world: pass or fail, rank, admission, a licence to practise, appointment to office; or, for formative tests, feedback that changes later learning

way in: Candidates arriving to sit

  1. L1: The pool of candidates who have prepared: in the late Qing two or three million applicants a year; in Song more than 400,000 registering for the prefectural examinations by the mid-13th century
  2. L2: Eligibility and identity at the door: registration, guards verifying identity and searching for hidden texts; identification where invigilators do not know the candidates
  3. L3: Seated under set conditions: numbered cells (Heavenly Stems and Earthly Branches), a hall, classroom or testing centre, fixed time
  4. L4: Sitting the paper: up to three days and two nights in a cell with no outside communication (Ming-Qing); a timed paper or computer test today

way in: Examiners, markers and invigilators

  1. L1: The pool of officials, teachers and paid scorers who set, watch and mark
  2. L2: Rules on who may examine whom: from 1037 no examiner in his home prefecture, no contact with high officials before the exams; instructors not invigilating their own tests
  3. L3: Watching the sitting and reading the scripts: invigilators to instruct and prevent cheating; markers reading against rubrics and benchmark papers
  4. L4: Judgement against the standard: first and second examining officials, a determining official; scorer agreement 60 to 85 percent, disagreements sent to more scorers

way in: Places, posts and quotas waiting on the result

  1. L1: What the result opens: offices (7,085 of 18,700 posts held by jinshi in 1046), university places, licences to practise
  2. L2: Quotas set ahead: degree quotas from 1009; Yuan quotas of 75 per group of four; prefectural quotas on a stepwise rule (8, 12, 15, 20)
  3. L3: The cycle: triennial provincial, metropolitan and palace examinations; annual entry-level examinations
  4. L4: The cut applied to the ranked list: the first n of n positions pass
  5. L5: Held stable over the dynasty and not raised with population, so the per-head chance fell

way in: Smuggled answers and doctored scripts (cheating)

  1. L1: Unauthorised sources: hidden copies of the classics, notes, a paid proxy, bribes to examiners, answers supplied by teachers
  2. L2: Brought past the door: texts written on underwear; a handkerchief of 10,000 characters
  3. L3: Into the answers: copying, or adults erasing wrong answers to right ones (89 percent of erasures at one school were incorrect to correct)
  4. L4: The doctored script reaches the marking: unexpected score swings and suspicious answer strings

way in: The body and conditions on the day (fatigue, anxiety, air)

  1. L1: Ambient conditions in the hall: the air breathed (PM2.5 on the exam day), the time of day the test falls
  2. L2: The candidate's body taking it in: cognitive fatigue building over the school day; physiological over-arousal (racing heart, stomach aches, sweating)
  3. L3: Arousal and appraisal: the worry and emotionality components of test anxiety; evaluation threat
  4. L4: Performance on the items moves with it: -0.9% of an SD per hour later; -8.2% of an SD on very polluted days; about 12 percentile points for highly anxious students

way in: The standard-setting panel (how a fixed cut is fixed)

  1. L1: Subject-matter experts representing the test's stakeholders (instructors, practitioners)
  2. L2: A panel of 5-15 convened by the test sponsor; a cut score 'cannot be arbitrarily determined', it must be empirically justified
  3. L3: A method: item-centred (Angoff, Ebel, Nedelsky, Bookmark) or person-centred (Borderline Survey, Contrasting Groups); performance-level descriptors written
  4. L4: Recommended cut scores assigning examinees to levels
  5. L5: The final decision is a policy decision by a policy body (a state board of education, a licensing authority)

way in: The syllabus or canon to be tested

  1. L1: A body of knowledge set down in texts: the Confucian canon, the Four Books with Zhu Xi's commentary; subject syllabi today
  2. L2: The authority narrows it into what will be examined: Ming authorities narrowed content to Neo-Confucian orthodoxy; examining bodies list command words
  3. L3: Turned into items and papers: test banks of up to four thousand questions; the eight-legged essay form; multiple variants of one test
  4. L4: The paper set before the candidate
  5. L5: The items checked against each other after the sitting: alpha computed for each concept, items with near-zero item-total correlation deleted or revised

way in: What rides on the score (stakes and incentives)

  1. L1: Consequences tied to scores for students, teachers and schools: promotion to the next grade, school probation and reconstitution, funding, pay
  2. L2: Bright-line rules: probation if under 15 percent at national norms; promotion cut-offs in grades 3, 6 and 8 (Chicago, 1996-97)
  3. L3: People respond: teaching to the test, more effort, special-education placements, retention, cheating
  4. L4: The scores shift: high-stakes gains that do not carry over to a low-stakes exam
  5. L5: The score loses value as an indicator and distorts what it was meant to monitor (Campbell)

way in: What the candidate knows, drawn out by the question

  1. L1: What the candidate has learned and holds in memory: years of study of a canon or syllabus
  2. L2: The item as a cue: a question, a set of answers to choose from, a blank to complete, an essay title with its command word
  3. L3: Retrieval from memory, which also strengthens what is retrieved (the testing effect); short-answer questions that can be answered from memory strengthen it, multiple-choice in one lecture study did not
  4. L4: The answer composed and recorded: written, typed, spoken or performed; paper and computer modes gave the same total scores in one field experiment

way out: Backwash: the test shaping teaching

  1. L1: Schools and teachers working to the test: what is not tested is not taught
  2. L3: Instruction narrowed to what will be tested; low-stakes subjects dropped
  3. L6: The test's content and form become the model of the subject

way out: Feedback into learning (formative use)

  1. L1: The learner's memory strengthened by having retrieved: retention up to 23 weeks later after short-answer review
  2. L6: Strengths and weaknesses identified; later learning activities modified
  3. L7: Assessment used to monitor learning and give ongoing feedback

way out: Results: pass lists, grades and the sorting of candidates

  1. L1: Candidates sorted into offices, universities, professions, or failure: 1-2 percent jinshi success under the Tang; 1 percent in the late Qing
  2. L6: The pass list published: graduates' names read aloud before the emperor and recorded; grades and transcripts today
  3. L7: The decision: who passes, in what rank

way out: The record kept

  1. L1: Written records held by state and schools, read later by employers, historians and admissions offices
  2. L6: Archival lists of graduates; the date of the jinshi degree as a key biographical datum; grades on a transcript
  3. L7: The result entered as a lasting record

Sources (36) published science

Exam (Wikipedia, via Wikimedia Enterprise snapshot 2026-11-01, rev 1379540068; redirected from Test (assessment))
Gymnasium (school) (Wikipedia, via Wikimedia Enterprise)
Imperial examination (Wikipedia, via Wikimedia Enterprise snapshot) (behind its links)
Greving S, Richter T (2018) Examining the testing effect in university teaching: retrievability and question format matter. Front Psychol 9:2412. doi:10.3389/fpsyg.2018.02412 (PMC6288371, CC BY) (behind its links)
Jacob BA (2002) Accountability, incentives and behavior: the impact of high-stakes testing in the Chicago Public Schools. NBER Working Paper 8968 (behind its links)
Steinmayr R, Crede J, McElvany N, Wirthwein L (2015/2016) Subjective well-being, test anxiety, academic achievement: testing for reciprocal effects. Front Psychol 6:1994. doi:10.3389/fpsyg.2015.01994 (PMC4705295, CC BY) (behind its links)
Lavy V, Ebenstein A, Roth S (2014) The long run human capital and economic consequences of high-stakes examinations. NBER Working Paper 20647 (behind its links)
Bai Y, Jia R (2014) Social mobility and revolution: the impact of the abolition of China's civil service exam. Working paper (published as 'Elite recruitment and political stability', Econometrica 84(2):677-733, 2016) (behind its links)
Standardized test (Wikipedia, via Wikimedia Enterprise snapshot) (behind its links)
Revelle W. Classical test theory and the measurement of reliability. Chapter 7 of An introduction to psychometric theory with applications in R (open draft, personality-project.org) (behind its links)
Tavakol M, Dennick R (2011) Making sense of Cronbach's alpha. Int J Med Educ 2:53-55. doi:10.5116/ijme.4dfb.8dfd (PMC4205511, CC BY) (behind its links)
Sievertsen HH, Gino F, Piovesan M (2016) Cognitive fatigue influences students' performance on standardized tests. PNAS 113(10):2621-2624. doi:10.1073/pnas.1516947113 (PMC4790980) (behind its links)
Test anxiety (Wikipedia, via Wikimedia Enterprise snapshot) (behind its links)
Jacob BA, Levitt SD (2002) Rotten apples: an investigation of the prevalence and predictors of teacher cheating. NBER Working Paper 9413 (published QJE 2003) (behind its links)
Academic dishonesty (Wikipedia, via Wikimedia Enterprise snapshot) (behind its links)
Chetty R, Deming DJ, Friedman JN (2023, revised 2025) Diversifying society's leaders? The determinants and causal effects of admission to highly selective private colleges. NBER Working Paper 31492 (behind its links)
Flore PC, Wicherts JM (2015) Does stereotype threat influence performance of girls in stereotyped domains? A meta-analysis. J Sch Psychol 53(1):25-44. doi:10.1016/j.jsp.2014.10.002 (behind its links)
Stoevenbelt AH, Flore PC, Wicherts JM et al. (2025) Registered Replication Report: Johns, Schmader, & Martens (2005). PsyArXiv doi:10.31234/osf.io/qctkp_v5 (behind its links)
Tamblyn R, Abrahamowicz M, Brailovsky C et al. (1998) Association between licensing examination scores and resource use and quality of care in primary care practice. JAMA 280(11):989-996. doi:10.1001/jama.280.11.989 (behind its links)
Bar examination (Wikipedia, via Wikimedia Enterprise snapshot) (behind its links)
Dee & Jacob 2009, The Impact of No Child Left Behind on Student Achievement, NBER Working Paper 15531 (behind its links)
Standard-setting study (Wikipedia, via Wikimedia Enterprise snapshot) (behind its links)
Equating (Wikipedia, via Wikimedia Enterprise snapshot) (behind its links)
Testing effect (Wikipedia, via Wikimedia Enterprise snapshot) (behind its links)
Boevé AJ, Meijer RR, Albers CJ, Beetsma Y, Bosker RJ (2015) Introducing computer-based testing in high-stakes exams in higher education: results of a field experiment. PLoS One 10(12):e0143616. doi:10.1371/journal.pone.0143616 (PMC4671535) (behind its links)
Campbell's law (encyclopedia; WME) (behind its links)
AERA, APA and NCME (2014) Standards for Educational and Psychological Testing. Washington DC: AERA (open access since 2021) (behind its links)
Grammar school (Wikipedia, via Wikimedia Enterprise) (behind its links)
Secondary education (Wikipedia, via Wikimedia Enterprise snapshot, rev 1379095735) (behind its links)
Grading in education (Wikipedia, via Wikimedia Enterprise snapshot) (behind its links)
Pennington CR, Heim D, Levy AR, Larkin DT (2016) Twenty years of stereotype threat research: a review of psychological mediators. PLoS One 11(1):e0146487. doi:10.1371/journal.pone.0146487 (PMC4713435) (behind its links)
Geiser S, Santelices MV (2007) Validity of high-school grades in predicting student success beyond the freshman year. UC Berkeley CSHE Research and Occasional Paper 6.07 (behind its links)
Carl Brigham (Wikipedia, via Wikimedia Enterprise snapshot) (behind its links)
Meritocracy (Wikipedia, via Wikimedia Enterprise snapshot) (behind its links)
Fu Shanxiang (Wikipedia, via Wikimedia Enterprise snapshot) (behind its links)
Signalling (economics) (Wikipedia, via Wikimedia Enterprise snapshot) (behind its links)