← all systems · see it in The System
Examination
An examination (exam or evaluation), or test, is an educational assessment intended to measure a test-taker's knowledge, skill, aptitude, physical fitness, or classification in many other topics (e.g., beliefs).
Properties published scienceour proposal
Sourced fields, and our own readings.
| aliases | Examinations and leaving certificates |
| born at | 165 BC (Han recruitment through examinations); written examinations in earnest under the Sui (605 or 607) |
| born precision | decades |
| boundary | Inside: the formal occasion of testing and its machinery: the tasks set (questions, items, essays, performances), the candidate answering under set conditions (place, time, invigilation, identity checks), the marking and scoring of the answers (anonymised papers, markers, rubrics, machines), the grade or rank read against a norm or a criterion, and the decision it opens (pass, rank, admission, licence, appointment). Outside: the learning before it (school, university, tutors, the candidate's own memory), the institutions that set the stakes and use the result (state and bureaucracy, universities, professional bodies, employers), and the candidate as a person (walked as Human). |
| nested level | A practice and institution of human society (rung 11): built from candidates, tasks, examiners and a scoring rule; a part of schools, universities, bureaucracies and licensing bodies (their door or their leaving gate). The family runs from informal oral tests to imperial civil-service examinations, written school exams, standardized tests and professional licensing. |
| scale | social |
Measures (35) published science
| examination cycle (characteristic clock) | triennial for the higher-level degrees (decreed 1067; nominal in practice); entry-level examinations every year years |
| length of one sitting (clock) | three sessions (Ming provincial and metropolitan), the second three days after the first and the third three days after that; each could take up to three days and two nights of writing |
| imperial examination lifetime | 1,300 years, to 1905 |
| Tang jinshi pass rate | 1-2 percent; 6,585 jinshi in the dynasty, about 23 a year |
| Song prefectural candidates per year | 100,000 (mid-12th century); more than 400,000 (mid-13th century) candidates |
| Song degree-holders from non-official families | about 57 percent (1148 and 1256 lists) |
| Ming metropolitan graduates from official families | 14 percent early Ming; about 60 percent late Ming |
| shengyuan per head | one per 300 people (1600); one per thousand (mid-19th century) ratio |
| Qing jinshi | 26,747 in 112 examinations, 1644-1905; 238.8 per examination degrees |
| late Qing pass rate | 1 percent of two or three million annual applicants |
| shengyuan to jinshi | one in three thousand ratio |
| average age of jinshi candidates (settling time) | thirties years of age |
| Tang Grand Chancellors from the examination | about 85 percent by the ninth century |
| entry-level quota steps | most common 8, 12, 15 and 20 per county or prefecture places |
| agreement between human scorers | 60 to 85 percent percent agreement |
| error variance at reliability 0.80 | 0.20 (true-score share 0.80), by the definition that reliability is the true-score share of score variance fraction of score variance |
| acceptable Cronbach's alpha | 0.70 to 0.95 (maximum 0.90 recommended) coefficient |
| time-of-day effect on test scores | -0.9% of an SD per hour later (95% CI 0.7-1.0%); a 20-30 min break +1.7% of an SD |
| pollution effect on exam scores | 1 SD of PM2.5 AQI (22.81) = -0.65 points (2.8% of an SD); AQI above 101 = -1.95 points (8.2% of an SD) Bagrut points |
| testing effect size (meta-analyses) | d/g 0.50 to 0.61 effect size |
| students with test anxiety | 10 to 40 percent |
| test-anxiety score gap | about 12 percentile points |
| classrooms with teacher cheating (Chicago) | at least 4-5 percent a year (3.4-5.6 percent across specifications) percent of classrooms |
| Chicago probation threshold | under 15 percent of students at or above national norms in reading |
| college students who have cheated | 50 to 70 percent at least once (US, since the 1960s studies) |
| Ivy-Plus attendance gap at equal scores | top-1% children more than twice as likely as middle-class children with comparable SAT/ACT scores ratio |
| stereotype threat effect (girls, MSSS tests) | d = -0.22 over 47 effect sizes; signs of publication bias |
| stereotype threat effect (registered replication) | d = -0.006 (N = 822) against the original d = -0.82 (N = 45) effect size |
| licensing score and care | +3.8 consultations, -2.7 inappropriate prescriptions, +6.6 mammography referrals per 1000 patients per SD of score |
| US state testing spend 2012 | $669 million; $27 per student (45 states) USD |
| China legal qualification exam pass rate | 10 to 15 percent a year (from 2018) |
| test bank size | up to four thousand questions items |
| Yuan provincial quota | 300 places, 75 per group; 28 Han Chinese from South China among them |
| NCLB effect on NAEP 4th-grade maths | effect size 0.22 by 2007; no evidence of an effect on reading in grades 4 or 8 |
| standard-setting panel size | 5-15 subject-matter experts people |
Open contentions (12) open contention
Where sources or readings disagree. Both stay on the map.
- C-3481 Do exam credentials certify skill or mainly signal ability?
booked: The map keeps the competing readings; none is ruled.
the other reading: human capital: the credential reflects skill learned; signalling and the sheepskin effect (encyclopedia, Signalling); Collins's credentialism (specialist's lead, not opened) - C-3504 Did China's examinations begin under the Han?
booked: The map keeps the competing readings; none is ruled.
the other reading: Han examinations did not offer an official avenue to appointment; most posts went by recommendation (encyclopedia, Exam); open competitive written examination begins with the Sui (605/607) - C-3507 Do high-stakes tests raise learning or only scores?
booked: The map keeps the competing readings; none is ruled.
the other reading: test scores provide a clear-cut, objective standard that checks grade inflation (supporters, encyclopedia); testing raises achievement (Phelps 2012, cited in the encyclopedia; not read); NCLB accountability raised 4th-grade maths on the low-stakes NAEP audit test (effect size 0.22 by 2007), not reading (Dee and Jacob 2009, abstract); in Chicago, high-stakes testing had a slight negative effect on the low-stakes IGAP once trends were controlled, though low-achieving schools gained more on both (Jacob 2002, full text) - C-3512 Did China's imperial examinations open office to ordinary families?
booked: The map keeps the competing readings; none is ruled.
the other reading: most did have some relative in the bureaucracy (encyclopedia); estimates criticised for ignoring larger kin networks (Bai and Jia's summary); late Ming about 60 percent of metropolitan graduates from established elite families; study was costly and most candidates came from the wealthy land-owning class (encyclopedia) - C-3515 Who invented the numerical mark?
booked: The map keeps the competing readings; none is ruled.
the other reading: Farish invented quantitative marking (Marlin); the evidence for Farish is unpersuasive (Stray) - C-3541 Were Western written examinations modelled on China's?
booked: The map keeps the competing readings; none is ruled.
the other reading: Europe had its own examinations from Bologna (1215 or 1219), oral and disputational, before Europeans knew of the Chinese system (1570); the first written examinations in Europe: Trinity College Cambridge 1702, or 1518 (Sadler, who admits 'evidence is not very clear'); Prussian medical examinations from 1725 and German examinations around 1800 (encyclopedia lists them without a Chinese link) - C-3552 How much of a test score is error when its reliability is 0.80?
booked: The map keeps the competing readings; none is ruled.
the other reading: 0.36 error variance at reliability 0.80, by squaring the reliability (Tavakol and Dennick 2011, as printed) - C-3559 Does stereotype threat lower test scores?
booked: The map keeps the competing readings; none is ruled.
the other reading: stereotype threat lowers test performance through anxiety, negative thinking and mind-wandering using up working memory (Pennington et al. 2016 review of 45 experiments); different forms of threat have different mediators; no one mediator unequivocally supported (Pennington et al. 2016) - C-3561 Does test anxiety cause low scores, or does poor preparation cause both?
booked: The map keeps the competing readings; none is ruled.
the other reading: interference / attentional control (Eysenck et al., via the encyclopedia); skill deficit: poor preparation produces both anxiety and low scores (encyclopedia; Tobias 1985 per the specialist, not opened) - C-3562 Does practice testing help memory with every kind of question?
booked: The map keeps the competing readings; none is ruled.
the other reading: one meta-analysis found multiple-choice practice gave stronger testing effects than short answer; another found the opposite (as reviewed by Greving and Richter) - C-3566 Do a test's consequences count against its validity?
booked: The map keeps the competing readings; none is ruled.
the other reading: consequences are an aspect of validity: washback and score corruption are validity questions (the specialist's reading of Messick 1989/1995; not opened) - C-3569 What do college admission tests predict?
booked: The map keeps the competing readings; none is ruled.
the other reading: scores may not be pure measures of merit: high-income families may buy test preparation (the same paper's footnote); tests offer narrow information; non-academic potential is not tested (encyclopedia, Standardized test); many US colleges have dropped test requirements (encyclopedia); in the University of California sample, high-school GPA was the strongest predictor of four-year college outcomes and had less adverse impact than standardized tests (Geiser and Santelices 2007, abstract)
Cross-scale patterns seen here (18) our proposal
Shapes that recur when the same kind of problem is solved at different scales, and how each shows in Examination. Each links to every other system that shows it.
- lightning (structural)
Years of stored preparation are discharged in one sitting whose outcome changes what follows: a transitory disturbance on the exam day (pollution) has permanent consequences for schooling and earnings (Lavy et al.); the abolition of 1905, a single edict, unlocked the revolution (Rozman: 'a more impo - nested ladder (structural)
The imperial examination as a ladder of implication: county, prefectural and college examinations (shengyuan), then provincial (juren), metropolitan and palace (jinshi); each degree requires the one below ('a roughly three-tiered ladder from local to provincial to court exams'). - same role, different builds (structural)
One job, choosing who holds office, done by different builds: recommendation (Han xiaolian, one candidate per 200,000 households), written competitive examination (Sui to Qing), patronage (Britain before 1855), sale of titles, hereditary privilege. And one job, judging what a scholar knows, done ora - symmetry breaking (name the cut) (structural)
The named cut: candidates made equal before the test (same paper, same conditions, names hidden, papers recopied) are split by rank at the quota or pass mark into passed and failed; the cut then sets each life's direction (office, licence, admission). - templating (structural)
One module transplanted between systems and still working: the Chinese competitive written examination was copied into the East India Company's Indian civil service (1829, 1855), then England's (1870), France's and America's (1883); in Parliament it was called 'the Chinese Principle'. Korea (958), V - The Payment to Entropy (structural)
Signal form: each hand-off of the candidate's knowledge loses part of it. The measuring step: reliability is the true-score share of score variance, so at 0.80 a fifth of the variance is error (corrected from 0.36 in the specialist pass; Tavakol and Dennick's printed figure kept as a contention). Th - the whole sets the local reference (structural)
In a norm-referenced test the cohort sets the reference each candidate is read against: 'Norm-referencing is usually called grading on a curve when the comparison group is students in the same classroom'; the same answer scores differently in a different cohort. - trade one property for another (structural)
Item formats trade scoring ease for depth: multiple-choice items are quick, cheap and reliable to score with wide coverage but cannot show knowledge beyond the choices; essays test complex objectives but are slow to score, subjective and 'may potentially be unreliable'; and alpha (reliability) is bo - Umwelt: each system carves its own slice of one shared medium (structural)
Each kind of examination carves its own slice of the same person: an IQ test, a bar exam, a driving test and a fitness test each read a different part; 'most academic tests (standardized or not) offer narrow information of achievement' and leave out interpersonal or soft skills. The examination's ow - who opens the door (structural)
The door is opened from the receiving side ahead of time: the state fixes posts, quotas and the cycle before anyone sits ('If the examination is open for n positions, then the first n candidates in ranks pass'); cells are pre-assigned by a numbering system; the licensing body keeps its register open - borrowed-then-held (partial)
The British civil service first ran the borrowed Chinese principle in India through the East India Company (college 1806, examinations 1829, open competition 1855), then held it at home (1870). Both phases are shown, but what was borrowed is a template, not a regulation the surroundings did for the - holding / entrainment (partial)
The examination's calendar sets the clock of its candidates' study: triennial higher examinations, annual entry-level ones; 'swotvac' revision weeks before finals; test times set by class schedules. The parts (candidates) tick to the whole's cycle; no source shows a candidate free-running when the c - red-queen (partial)
Cheaters and examiners both improve: anonymity (702, 992-1032), recopying (1015, 1037), separation of examiners (1037) and body searches against hidden texts, microscopic cheat handkerchiefs and bribery; neither side ends the contest (Yangzhou 1711; modern cheating rates stable since the 1960s). The - same toolkit, independent builds (partial)
Bologna's oral examinations (1215 or 1219) were built from questions, examiners and a degree, before Europe knew of the Chinese system (1570); China built its examinations from the same parts. Korea, Vietnam and Japan are not independent (copied from China). - The carry (partial)
Overflow and the wait: places are capped by quotas; the excess of candidates is passed on to the next cycle (triennial), so the average jinshi was in his thirties and one shengyuan in three thousand ever became one; the overflow of degree-holders without office passed into local roles as teachers, p - causal ceiling (surface)
Reliability 'reveals the effect of measurement error on the observed score of a student cohort rather than on an individual student'; individual scores may be too noisy while group means inform. A limit on what one score can tell about one person, set by the test's error structure; but longer tests - conservation law plus closure (surface)
Alpha leaves the observed score's true and error parts open until an assumption closes it: alpha 'is grounded in the tau equivalent model', and violating it underestimates reliability. The decomposition itself is not quoted in the source read. - not-diluting wins (surface)
As quotas held while population grew, shengyuan per head diluted (one per 300 to one per thousand), while the jinshi kept their hold on the higher offices ('still dominated by jinshi degree-holders'). A scarce degree holding its worth, read loosely as not diluting.
Valence: what it trades, and with whom (14) published science
Each exchange with a neighbour, read from both sides: who gains, who pays, and what carries it.
- Human (Homo sapiens) via sys:human-society · Examination gain
- State (polity); the nation state its predominant modern form via sys:human-society · Examination gain
- University via sys:human-society · Examination gain
- Family (the human family as a system) via sys:human-society · Examination even
- Bureaucracy via sys:human-society · Examination gain
- Literacy via sys:written-record · Examination gain
- Memory via sys:memory · Examination gain
- 1911 Revolution (Xinhai) via sys:human-society · Examination loss
- Curriculum via sys:written-record · Examination gain
- Teacher via sys:human-society · Examination gain
- Secondary school via sys:human-society · Examination gain
- Physician via sys:human-society · Examination gain
- Air pollution via sys:sensory-environment · Examination loss
- Adolescence via sys:human-society · Examination even
Partners and parts published science
Built from / builds (9)
- Secondary school
- Human (Homo sapiens)
- Tasks and items (questions, essays, performances)
- Examiners, markers and invigilators
- Score, grade and the scale read against a norm or criterion
- The examination hall, cells and test centre
- Quotas and pass marks
- Standard-setting panel (cut score)
- Score scale, linking and equating (the measurement model)
Sits inside / holds (5)
expressed by (16)
- Templating / Standardization
- Internal Coordination/Regulation
- Sensing/Scanning
- Uncertainty Reduction/Learning
- Trail-Marking / Externalized Memory (Stigmergy)
- Subversion & Co-optation
- Physical/Virtual Barrier
- Technique-Based Identification (a.k.a. Technique-Based Detection)
- Self-Monitoring
- Validation of Identity/Intention
- Validation of Effectiveness
- Deception
- Redundancy & Diversity
- Triage & Prioritized Allocation
- Deterrence
- Quality Control & Error Correction
floor filled by (16)
- Secondary school
- Human (Homo sapiens)
- State (polity); the nation state its predominant modern form
- University
- Written record (durable external marks: texts, archives, canon)
- Family (the human family as a system)
- Bureaucracy
- Memory
- Curriculum
- Psychometrics
- Eight-legged essay
- Tasks and items (questions, essays, performances)
- Examiners, markers and invigilators
- Score, grade and the scale read against a norm or criterion
- The examination hall, cells and test centre
- Quotas and pass marks
represents (1)
Kinds (24)
- The imperial examination system
- Standardized test
- Public disputation
- Intelligence quotient
- Merit recruitment (examinations)
- Formative assessment
- Summative assessment
- Norm-referenced test
- Criterion-referenced test
- High-stakes test
- Competitive examination
- Licensing examination
- Bar examination
- Oral examination
- Written examination
- Performance test
- Entrance examination
- Final examination
- Open-book examination
- Large-scale survey assessment (PISA, NAEP)
- Eleven-plus examination
- Matura / Abitur
- GCE O level and A level
- PISA
wired (1)
Notes published science
- To stop examiners recognising a favourite's handwriting, Song China had every answer paper recopied by clerks before it was marked, from 1015; under the Ming and Qing each essay was copied three times and read by two examiners before a third fixed the grade, while candidates smuggled in handkerchiefs carrying 10,000 characters of the classics in microscopic script (src:wp:imperial-examination).
- When China abolished its 1,300-year-old examination in 1905, the places that had been given the most exam places per head sent the most men into the revolution that ended imperial rule six years later: a revolutionary leader asked, 'if the exam were not abolished, who would have followed our revolution?' (src:bai-jia-2014).
- A bad-air day on the day of Israel's high-school matriculation exams lowered students' scores, and the students who sat on those days went on to less university schooling and lower adult earnings: one random morning's air, written into a life through the exam (src:lavy-2014).
Layers
The ways it takes in its world, where they meet, and how it acts back, each from L1 (where it meets the world) up.
examination: from what a candidate has learned to a score and the decision it opens (reader's view)
- L1: The examination's world: candidates who have prepared (in schools, with tutors and cram schools, for years), the body of knowledge or skill a syllabus sets down, and the offices, places and licences waiting on a decision
- L2: The door: registration, eligibility and identity; guards searching for hidden texts; invigilators present to instruct and to prevent cheating
- L3: The task: items and papers drawn from the syllabus (multiple choice, completion, essay, oral question, performance); set forms such as the eight-legged essay; tiers of examinations from local to provincial to court
- L4: The response under set conditions: the candidate answering in the hall or cell, on paper, computer, by voice or by performing; fatigue, anxiety and the air on the day enter here
- L5: Marking and scoring: anonymised and recopied papers, several markers and a determining official, uniform scoring, rubrics, machine scoring, statistical checks; reliability
- L6: Meaning: the score or grade read against a norm or a criterion, its reliability and validity; the quota or pass mark that sets the cut; validity is judged for the interpretation and use, not the test; cut scores come from a quota or from a standard-setting panel and a policy decision
- L7: The decision acting on the world: passing, ranking, admitting, licensing, appointing; feedback into learning; and backwash on what is taught
the boardroom (shared floors)
- L5: Marking and scoring: anonymised and numbered papers recopied three times, two markers and a determining official (imperial China); uniform administration and scoring, rubrics, double scoring, machine scoring; statistical checks for cheating; reliability estimated (alpha 0.70-0.95 acceptable); score linking and equating across forms; differential item functioning screens and accommodations (fairness as a validity issue). Alpha, the usual estimate, is Guttman's lambda-3, a lower bound that many psychometricians deplore relying on alone (Revelle)
- L6: What the score means for a use: validity is a property of the interpretation of scores for a proposed use, not of the test, and validation is an argument for and against that interpretation (Standards); threats are construct underrepresentation and construct-irrelevant variance; reliability caps it; read against a norm (rank, curve) or a criterion fixed by a standard-setting panel and adopted by a policy body, or against a quota; bent by the stakes riding on it
- L7: The decision acting on the world: pass or fail, rank, admission, a licence to practise, appointment to office; or, for formative tests, feedback that changes later learning
way in: Candidates arriving to sit
- L1: The pool of candidates who have prepared: in the late Qing two or three million applicants a year; in Song more than 400,000 registering for the prefectural examinations by the mid-13th century
- L2: Eligibility and identity at the door: registration, guards verifying identity and searching for hidden texts; identification where invigilators do not know the candidates
- L3: Seated under set conditions: numbered cells (Heavenly Stems and Earthly Branches), a hall, classroom or testing centre, fixed time
- L4: Sitting the paper: up to three days and two nights in a cell with no outside communication (Ming-Qing); a timed paper or computer test today
way in: Examiners, markers and invigilators
- L1: The pool of officials, teachers and paid scorers who set, watch and mark
- L2: Rules on who may examine whom: from 1037 no examiner in his home prefecture, no contact with high officials before the exams; instructors not invigilating their own tests
- L3: Watching the sitting and reading the scripts: invigilators to instruct and prevent cheating; markers reading against rubrics and benchmark papers
- L4: Judgement against the standard: first and second examining officials, a determining official; scorer agreement 60 to 85 percent, disagreements sent to more scorers
way in: Places, posts and quotas waiting on the result
- L1: What the result opens: offices (7,085 of 18,700 posts held by jinshi in 1046), university places, licences to practise
- L2: Quotas set ahead: degree quotas from 1009; Yuan quotas of 75 per group of four; prefectural quotas on a stepwise rule (8, 12, 15, 20)
- L3: The cycle: triennial provincial, metropolitan and palace examinations; annual entry-level examinations
- L4: The cut applied to the ranked list: the first n of n positions pass
- L5: Held stable over the dynasty and not raised with population, so the per-head chance fell
way in: Smuggled answers and doctored scripts (cheating)
- L1: Unauthorised sources: hidden copies of the classics, notes, a paid proxy, bribes to examiners, answers supplied by teachers
- L2: Brought past the door: texts written on underwear; a handkerchief of 10,000 characters
- L3: Into the answers: copying, or adults erasing wrong answers to right ones (89 percent of erasures at one school were incorrect to correct)
- L4: The doctored script reaches the marking: unexpected score swings and suspicious answer strings
way in: The body and conditions on the day (fatigue, anxiety, air)
- L1: Ambient conditions in the hall: the air breathed (PM2.5 on the exam day), the time of day the test falls
- L2: The candidate's body taking it in: cognitive fatigue building over the school day; physiological over-arousal (racing heart, stomach aches, sweating)
- L3: Arousal and appraisal: the worry and emotionality components of test anxiety; evaluation threat
- L4: Performance on the items moves with it: -0.9% of an SD per hour later; -8.2% of an SD on very polluted days; about 12 percentile points for highly anxious students
way in: The standard-setting panel (how a fixed cut is fixed)
- L1: Subject-matter experts representing the test's stakeholders (instructors, practitioners)
- L2: A panel of 5-15 convened by the test sponsor; a cut score 'cannot be arbitrarily determined', it must be empirically justified
- L3: A method: item-centred (Angoff, Ebel, Nedelsky, Bookmark) or person-centred (Borderline Survey, Contrasting Groups); performance-level descriptors written
- L4: Recommended cut scores assigning examinees to levels
- L5: The final decision is a policy decision by a policy body (a state board of education, a licensing authority)
way in: The syllabus or canon to be tested
- L1: A body of knowledge set down in texts: the Confucian canon, the Four Books with Zhu Xi's commentary; subject syllabi today
- L2: The authority narrows it into what will be examined: Ming authorities narrowed content to Neo-Confucian orthodoxy; examining bodies list command words
- L3: Turned into items and papers: test banks of up to four thousand questions; the eight-legged essay form; multiple variants of one test
- L4: The paper set before the candidate
- L5: The items checked against each other after the sitting: alpha computed for each concept, items with near-zero item-total correlation deleted or revised
way in: What rides on the score (stakes and incentives)
- L1: Consequences tied to scores for students, teachers and schools: promotion to the next grade, school probation and reconstitution, funding, pay
- L2: Bright-line rules: probation if under 15 percent at national norms; promotion cut-offs in grades 3, 6 and 8 (Chicago, 1996-97)
- L3: People respond: teaching to the test, more effort, special-education placements, retention, cheating
- L4: The scores shift: high-stakes gains that do not carry over to a low-stakes exam
- L5: The score loses value as an indicator and distorts what it was meant to monitor (Campbell)
way in: What the candidate knows, drawn out by the question
- L1: What the candidate has learned and holds in memory: years of study of a canon or syllabus
- L2: The item as a cue: a question, a set of answers to choose from, a blank to complete, an essay title with its command word
- L3: Retrieval from memory, which also strengthens what is retrieved (the testing effect); short-answer questions that can be answered from memory strengthen it, multiple-choice in one lecture study did not
- L4: The answer composed and recorded: written, typed, spoken or performed; paper and computer modes gave the same total scores in one field experiment
way out: Backwash: the test shaping teaching
- L1: Schools and teachers working to the test: what is not tested is not taught
- L3: Instruction narrowed to what will be tested; low-stakes subjects dropped
- L6: The test's content and form become the model of the subject
way out: Feedback into learning (formative use)
- L1: The learner's memory strengthened by having retrieved: retention up to 23 weeks later after short-answer review
- L6: Strengths and weaknesses identified; later learning activities modified
- L7: Assessment used to monitor learning and give ongoing feedback
way out: Results: pass lists, grades and the sorting of candidates
- L1: Candidates sorted into offices, universities, professions, or failure: 1-2 percent jinshi success under the Tang; 1 percent in the late Qing
- L6: The pass list published: graduates' names read aloud before the emperor and recorded; grades and transcripts today
- L7: The decision: who passes, in what rank
way out: The record kept
- L1: Written records held by state and schools, read later by employers, historians and admissions offices
- L6: Archival lists of graduates; the date of the jinshi degree as a key biographical datum; grades on a transcript
- L7: The result entered as a lasting record
Sources (36) published science
Imperial examination (Wikipedia, via Wikimedia Enterprise snapshot) (behind its links)
Jacob BA (2002) Accountability, incentives and behavior: the impact of high-stakes testing in the Chicago Public Schools. NBER Working Paper 8968 (behind its links)
Lavy V, Ebenstein A, Roth S (2014) The long run human capital and economic consequences of high-stakes examinations. NBER Working Paper 20647 (behind its links)
Standardized test (Wikipedia, via Wikimedia Enterprise snapshot) (behind its links)
Tavakol M, Dennick R (2011) Making sense of Cronbach's alpha. Int J Med Educ 2:53-55. doi:10.5116/ijme.4dfb.8dfd (PMC4205511, CC BY) (behind its links)
Test anxiety (Wikipedia, via Wikimedia Enterprise snapshot) (behind its links)
Jacob BA, Levitt SD (2002) Rotten apples: an investigation of the prevalence and predictors of teacher cheating. NBER Working Paper 9413 (published QJE 2003) (behind its links)
Academic dishonesty (Wikipedia, via Wikimedia Enterprise snapshot) (behind its links)
Stoevenbelt AH, Flore PC, Wicherts JM et al. (2025) Registered Replication Report: Johns, Schmader, & Martens (2005). PsyArXiv doi:10.31234/osf.io/qctkp_v5 (behind its links)
Bar examination (Wikipedia, via Wikimedia Enterprise snapshot) (behind its links)
Dee & Jacob 2009, The Impact of No Child Left Behind on Student Achievement, NBER Working Paper 15531 (behind its links)
Standard-setting study (Wikipedia, via Wikimedia Enterprise snapshot) (behind its links)
Equating (Wikipedia, via Wikimedia Enterprise snapshot) (behind its links)
Testing effect (Wikipedia, via Wikimedia Enterprise snapshot) (behind its links)
Campbell's law (encyclopedia; WME) (behind its links)
AERA, APA and NCME (2014) Standards for Educational and Psychological Testing. Washington DC: AERA (open access since 2021) (behind its links)
Grammar school (Wikipedia, via Wikimedia Enterprise) (behind its links)
Secondary education (Wikipedia, via Wikimedia Enterprise snapshot, rev 1379095735) (behind its links)
Grading in education (Wikipedia, via Wikimedia Enterprise snapshot) (behind its links)
Geiser S, Santelices MV (2007) Validity of high-school grades in predicting student success beyond the freshman year. UC Berkeley CSHE Research and Occasional Paper 6.07 (behind its links)
Carl Brigham (Wikipedia, via Wikimedia Enterprise snapshot) (behind its links)
Meritocracy (Wikipedia, via Wikimedia Enterprise snapshot) (behind its links)
Fu Shanxiang (Wikipedia, via Wikimedia Enterprise snapshot) (behind its links)
Signalling (economics) (Wikipedia, via Wikimedia Enterprise snapshot) (behind its links)