← all systems · see it in The System
Large language model
To discriminate the difference in parameter scale, the research community has coined the term large language models (LLM) for the PLMs of significant size. Its basic function is to generate statistically likely continuations of word sequences (Shanahan 2022); today it is used inside dialogue systems and agents with tools, retrieval and system instructions.
Properties published scienceour proposal
Sourced fields, and our own readings.
| born at | 2020 (GPT-3, 175B parameters; the survey names it and PaLM as early LLMs) |
| born precision | the field draws no sharp line; by parameter scale |
| boundary | Inside: the trained network (tokenizer, embeddings, transformer blocks, unembedding, weights) and its run over a context, from the prompt to the sampled token; in today's deployed form also the instruction, tool, retrieval and long-context arrangements that feed it. Outside: the people, software, corpus and hardware it exchanges with. Shanahan keeps 'the bare-bones LLM' apart from 'the systems in which they are embedded' (contention C-431). |
| nested level | Built from transformer components (token embedding, attention heads, MLP layers, unembedding communicating through a residual stream); embedded as the core computation unit of dialogue systems and LLM-based agents; runs on accelerators in datacentres; its medium is written language. |
| scale | technological |
Measures (17) published science
| GPT-3 parameters | 175 billion parameters |
| GPT-3 training tokens | 300 billion tokens |
| Common Crawl size (GPT-3 corpus) | nearly a trillion words |
| Transformer base: layers and width | N = 6 layers; dmodel = 512 layers; dimensions |
| BPE vocabulary (WMT 2014 En-De) | about 37000 tokens |
| characteristic clock: one token per decoding step | 1 token per forward step |
| characteristic clock: Transformer (big) training run, En-Fr | 3.5 days on eight GPUs |
| characteristic clock: model generations | 1 generation (a model trained on the previous generation's output) |
| power-law range of loss scaling | more than seven orders of magnitude |
| Chinchilla: parameters and data | 70B parameters, 4x the data of Gopher (same compute) parameters; relative tokens |
| InstructGPT 1.3B preferred over GPT-3 175B | 100x fewer parameters, yet preferred |
| labelers (InstructGPT) | about 40 contractors |
| context window (GPT-4 Turbo) | up to 128k tokens |
| image-text pairs (CLIP pre-training) | 400 million pairs |
| carbon footprint range from DNN, datacentre and processor choice | ~100-1000 times |
| NLP community survey on LM understanding | 51% agree / 49% disagree (480 respondents) percent |
| ReAct gain on ALFWorld / WebShop | 34 / 10 absolute % success rate over imitation and RL baselines |
Open contentions (3) open contention
Where sources or readings disagree. Both stay on the map.
- C-432 Do large language models understand what they say?
booked: The map describes how these models work in the field's own terms (learned features, next-word probabilities) and rules nothing true or false about understanding, belief or meaning. The competing views are listed side by side.
the other reading: category error: models learn the form of language, not its meaning; no experience or mental models of the world (one faction, Mitchell & Krakauer); the networks truly understand language and reason in a general way, 'not yet' at human level (other faction); huge systems of correlations may be functionally equivalent to understanding (Mitchell & Krakauer's third question); Shanahan: a bare LLM does next-token prediction; beliefs make sense, if anywhere, only for a larger embedding system, and not for a simple dialogue agent; survey of NLP researchers: 51% / 49% - C-433 Do new abilities appear suddenly in language models as they grow, or gradually?
booked: The map records that abilities rise with scale and does not rule whether some appear suddenly or smoothly.
the other reading: Wei et al.: emergent abilities cannot be predicted by extrapolating smaller models; Schaeffer et al.: apparent emergence comes from nonlinear or discontinuous metrics - C-434 Could large language models be conscious?
booked: The map makes no claim about consciousness in these systems and lists the views side by side.
the other reading: Butlin et al.: 'no current AI systems are conscious' by indicator properties from neuroscientific theories; Butlin et al.: 'no obvious technical barriers to building AI systems which satisfy these indicators'; Shanahan: loaded words invite anthropomorphism
Partners and parts published science
Built from / builds (9)
- Tokenizer (subword vocabulary)
- Token embedding
- Residual stream
- Attention head
- Feed-forward (MLP) layer
- Unembedding and softmax
- LLM-based agent
- AI-generated and human-written poems for non-expert readers' preference
- Human-written and AI-generated essays rated against each other
Sits inside / holds (2)
Trades with (valence) (60)
- Grammar (the structure of a language, as linguistics describes it) via sys:written-record · Large language model gain
- Pragmatics via sys:written-record · Large language model gain
- Writing (the practice of marking language on a surface, and the script plus rules it works with) via sys:written-record · Large language model gain
- Vector calculus (the calculus of fields: gradient, divergence, curl; the practice and the structures it works on) via sys:written-record · Large language model gain
- Software via sys:written-record · Large language model gain
- Internet via sys:written-record · Large language model gain
- Chemistry (the field: the practice of chemists and the structure of substances and their reactions it works on) via sys:written-record · Large language model gain
- Vector space: sets whose elements can be added and scaled, as mathematics describes it, with bases and dimension, linear maps, inner products, and their use as the state spaces of physics and data via sys:computer-memory · Large language model gain
- Human (Homo sapiens) via sys:written-record · Large language model gain
- Language (human language) via sys:written-record · Large language model gain
- Science (as a human social and cultural system) via sys:written-record · Large language model mixed
- Data centre via sys:computer-hardware · Large language model gain
- LLM-based agent via sys:written-record · Large language model even
- Syntax: how a language combines words into phrases and sentences (constituency and dependency, word order, agreement, recursion), how speakers build and parse it, how children learn it, how it differs across languages, and the field that studies it via sys:acoustic-wave · Large language model gain
- Recursion: defining or computing a thing in terms of smaller versions of itself, as computer science and mathematics describe it, with its base cases, call stack, recursive data, induction, and its sightings in language and nature via sys:written-record · Large language model mixed
- Consciousness via sys:written-record · Large language model even
- Arithmetic (the practice of calculating, and the operations and laws it works with) via sys:written-record · Large language model gain
- Algorithm (the practice of step-by-step procedures and the structure they compute) via sys:written-record · Large language model gain
- Programming language via sys:written-record · Large language model gain
- Linear algebra (the practice of solving, transforming and taking apart linear systems with vectors and matrices, and the structures it works with) via sys:sensory-environment · Large language model gain
- Personality via sys:sensory-environment · Large language model gain
- Operant conditioning via sys:written-record · Large language model gain
- Psychology (the mind and behaviour as psychology describes them) via sys:sensory-environment · Large language model even
- Function (mathematics): the assignment of exactly one output to each input, and the practice of defining, drawing, composing and using it via sys:sensory-environment · Large language model gain
- Trigonometry (the practice of tying angles to lengths, and the structure it works on) via sys:written-record · Large language model gain
- Software engineering via sys:written-record · Large language model gain
- Poetry via sys:written-record · Large language model gain
- Rhetoric (speech and argument) via sys:written-record · Large language model gain
- Artificial intelligence via sys:written-record · Large language model gain
- Lyric poetry via sys:written-record · Large language model gain
- Argument: premises offered in support of a conclusion, and argumentation, the practice of giving and asking for reasons in doubt or disagreement, as philosophy and argumentation theory describe them via sys:written-record · Large language model even
- Phishing via sys:mental-representation · Large language model even
- Logic: the study and practice of correct reasoning (formal and informal, across its traditions) and the valid inference it works on, together with how people actually reason via sys:written-record · Large language model mixed
- Matrix (mathematics): the rectangular array of numbers and its algebra, determinants and eigenvalues, matrices as linear maps, their history, and their use in physics, graphics and machine learning; the practice and the structure both booked via sys:written-record · Large language model gain
- Computer hardware via sys:electromagnetic-field · Large language model gain
- Database via sys:written-record · Large language model gain
- Cloud computing via sys:electromagnetic-field · Large language model gain
- Secure software development via sys:software · Large language model gain
- Security information and event management (SIEM) via sys:written-record · Large language model gain
- Cyber threat intelligence via sys:written-record · Large language model even
- Social engineering via sys:written-record · Large language model even
- Penetration testing via sys:software · Large language model mixed
- Historiography via sys:written-record · Large language model even
- Information warfare via sys:social-network · Large language model even
- Morphology (linguistics): how a language builds its words from smaller meaningful parts (roots, affixes, compounding, inflection and derivation), how languages differ in this, how speakers learn and use it, and the field that studies it via sys:written-record · Large language model gain
- English language: a West Germanic language of the Indo-European family, from its Old English beginnings to its varieties and its use as a global lingua franca, as linguistics and its history describe it via sys:written-record · Large language model mixed
- Linguistics (the field: the practice and the languages it studies) via sys:written-record · Large language model gain
- Semantics: meaning in language (the meanings of words and sentences, sense and reference, how meanings combine, how they are held in minds and brains and shared in a community) and the field of linguistics and philosophy that studies it via sys:written-record · Large language model gain
- Curriculum via sys:written-record · Large language model even
- Old English literature via sys:written-record · Large language model gain
- Computer science (the field: the practice and the computation it studies) via sys:sensory-environment · Large language model gain
- Middle English literature via sys:written-record · Large language model even
- Mathematical proof: the practice of showing that a statement follows from axioms and earlier theorems, how it is written, checked (by people and machines) and accepted, and the derivations it works on via sys:written-record · Large language model gain
- Mathematical logic: the field that studies logic within mathematics (model theory, proof theory, set theory, computability theory) and the formal systems, structures and limit results it works on via sys:written-record · Large language model gain
- Combinatorics: the practice of counting, arranging and choosing finite structures (permutations, combinations, partitions, designs, graphs), the structures it studies, and the people and machines who work with them via sys:computer-hardware · Large language model even
- Financial statement via sys:written-record · Large language model gain
- Plato via sys:written-record · Large language model gain
- Voting via sys:written-record · Large language model gain
- Truth via sys:written-record · Large language model mixed
- Essay via sys:written-record · Large language model gain
expressed by (10)
- Generation/Synthesis
- Interoperability/Transduction (Bridging)
- Sensing/Scanning
- Search/Foraging
- Uncertainty Reduction/Learning
- Multi-Scale Data Fusion / Triangulation
- Frontier Outposts / Sensing Extension
- Social Learning / Public Information Use
- Validation of Identity/Intention
- Quality Control & Error Correction
floor filled by (41)
- Grammar (the structure of a language, as linguistics describes it)
- Writing (the practice of marking language on a surface, and the script plus rules it works with)
- Vector calculus (the calculus of fields: gradient, divergence, curl; the practice and the structures it works on)
- Software
- Chemistry (the field: the practice of chemists and the structure of substances and their reactions it works on)
- Vector space: sets whose elements can be added and scaled, as mathematics describes it, with bases and dimension, linear maps, inner products, and their use as the state spaces of physics and data
- Human (Homo sapiens)
- Science (as a human social and cultural system)
- Internet
- Stored-program principle (instructions held in the same memory as data)
- LLM-based agent
- Tokenizer (subword vocabulary)
- Token embedding
- Residual stream
- Attention head
- Feed-forward (MLP) layer
- Unembedding and softmax
- Vision encoder (CLIP ViT)
- Speech and audio encoders
- Data centre
- Learned feature (interpretability)
- Reward / preference model
- ML accelerator (GPU/TPU)
- Arithmetic (the practice of calculating, and the operations and laws it works with)
- Programming language
- Linear algebra (the practice of solving, transforming and taking apart linear systems with vectors and matrices, and the structures it works with)
- Trigonometry (the practice of tying angles to lengths, and the structure it works on)
- Software engineering
- Artificial intelligence
- Logic: the study and practice of correct reasoning (formal and informal, across its traditions) and the valid inference it works on, together with how people actually reason
- Matrix (mathematics): the rectangular array of numbers and its algebra, determinants and eigenvalues, matrices as linear maps, their history, and their use in physics, graphics and machine learning; the practice and the structure both booked
- Cloud computing
- Secure software development
- Cyber threat intelligence
- Social engineering
- Penetration testing
- Linguistics (the field: the practice and the languages it studies)
- Old English literature
- Mathematical logic: the field that studies logic within mathematics (model theory, proof theory, set theory, computability theory) and the formal systems, structures and limit results it works on
- Combinatorics: the practice of counting, arranging and choosing finite structures (permutations, combinations, partitions, designs, graphs), the structures it studies, and the people and machines who work with them
- Essay
Kinds (8)
- Language model
- Psychology (the mind and behaviour as psychology describes them)
- Artificial intelligence
- Causal decoder (decoder-only) LLM
- Prefix decoder LLM
- Encoder-decoder LLM
- Multimodal LLM
- Sparse mixture-of-experts LLM
wired (6)
- Writing (the practice of marking language on a surface, and the script plus rules it works with)
- Human (Homo sapiens)
- Human brain
- Decision procedure (agent program)
- Context window / working memory (main context)
- Book
Notes published science
- Interpretability researchers describe the residual stream as a channel that 'doesn't do any processing itself and all layers communicate through it' (Elhage et al. 2021): in this walk it is literally the board, the one place every incoming channel meets.
- One mechanism, two signs: model output fed back into training. Ranked by people (RLHF), it pulls outputs toward what labelers prefer; unranked and scraped back off the web, it makes later models lose 'the tails of the original content distribution' (Shumailov et al., 'model collapse').
- LLaVA's door for images is a single trainable projection matrix that turns image features into tokens 'which have the same dimensionality as the word embedding space': the port is built by training, on the receiving side, before any picture arrives.
Layers
The ways it takes in its world, where they meet, and how it acts back, each from L1 (where it meets the world) up.
electricity into computation (data centre)
- L1: Electrical energy drawn by training and inference
- L2: ML-oriented accelerators (GPUs, TPUs) inside datacentres doing the arithmetic
text in the context window (reader's view)
- L1: Text arriving as the prompt: the user's words, an application developer's system prompt, few-shot demonstrations, tool and retrieval results, placed in one input sequence (the context window)
- L2: Tokenizer: text cut into subword units from a fixed vocabulary (byte-pair encoding and kin)
- L3: Learned token embeddings: each token becomes a vector of the model's width; image and audio features are projected into the same space
- L4: The residual stream: the running sum of the embedding and every earlier layer's output, a channel every layer reads from and writes to
- L5: Attention heads and MLP (feed-forward) layers in a stack of identical blocks: heads move information between token positions (masked to the past in causal decoders); feed-forward layers act as key-value memories
- L6: Features: directions in activation space found by sparse autoencoders, many multilingual, multimodal and abstract (packed in superposition, so single 'neurons' are often polysemantic)
- L7: Next-token probabilities from the unembedding and softmax; a token is sampled and becomes text, a tool call or code; in an agent this floor is where plans and actions are produced
the boardroom (shared floors)
- L3: Token embeddings: every channel's tokens become vectors of one width; visual and audio features are projected into the same word-embedding space
- L4: The residual stream: one shared channel all layers read from and write to
- L5: Attention heads (moving information between positions, masked to the past) and feed-forward layers (key-value memories), in a stack of identical blocks
- L6: Features: concept-like directions in activation space, multilingual and multimodal, readable and steerable by interpretability methods
- L7: The output distribution over next tokens and the sampled token: what the model does to its context and, through outlets, to the world
training: the corpus into the weights
- L1: The corpus: webpages, books, conversational text and specialised data (code, science)
- L2: Mixing and processing: ready-made datasets combined, filtered, de-duplicated and tokenized
- L3: The language-modelling objective: predict each token from the ones before it
- L4: Weights updated: loss falls as a power law in model size, data and compute; factual knowledge is stored in the parameters
- L5: Fine-tuning on demonstrations, then reinforcement learning from human (or AI) preference rankings
way in: AI feedback (RLAIF / Constitutional AI)
- L1: Self-critiques, revisions and a model's choice between two samples, under a list of principles
- L2: A preference model trained on AI preferences, used as the reward signal
way in: Audio (speech, sounds, music)
- L1: General auditory information: speech, audio events and music
- L2: Speech and audio encoders joined to a text-based LLM
way in: Human feedback (RLHF)
- L1: Demonstrations of desired behaviour and rankings of model outputs by labelers
- L2: A reward model trained to predict which output labelers prefer; the model is tuned against it (PPO)
way in: Images
- L1: An input image given with the text
- L2: CLIP visual encoder (ViT-L/14), then a trainable projection into word-embedding space as visual tokens
way in: Own previous output (autoregression, reasoning traces)
- L1: Previously generated tokens, and in agents the model's own reasoning traces, appended to the context
way in: Retrieved passages
- L1: Non-parametric memory: a dense vector index of documents (Wikipedia in RAG)
- L2: A pre-trained neural retriever picks passages; the generator conditions on them
way in: System / developer instructions
- L1: System prompts: text from an application developer, placed ahead of the user's text
- L2: Same tokenizer; priority over other text comes from training, not from a separate door
way in: The user's message
- L1: Prompts written by people and submitted through an API or chat interface
- L2: Tokenizer: subword units from a fixed vocabulary (BPE)
way in: Tool results
- L1: Results of API calls (calculator, Q&A system, search engines, translation, calendar) and observations from an environment, read back into the context
way in: Training corpus (the slow channel)
- L1: Webpages, books, conversational text and specialised data
- L2: Mixed, processed and tokenized; then predicted token by token and the weights updated
way out: Code that runs
- L1: A runtime such as a Python interpreter executes them
- L7: Programs written as intermediate reasoning steps
way out: Text out
- L1: Text returned to the user by the dialogue system that wraps the model; some of it goes online, where later corpora are scraped
- L7: Sampling from the nucleus of the next-token distribution
way out: Tool / API calls
- L1: Tools run: calculator, Q&A system, search engines, translation system, calendar; in agents, actions in an environment
- L7: Deciding which API to call, when, and with what arguments
Sources (82) published science
Vaswani et al. 2017, Attention Is All You Need (arXiv:1706.03762) (behind its links)
Yao et al. 2022, ReAct: Synergizing Reasoning and Acting in Language Models (arXiv:2210.03629) (behind its links)
Ouyang et al. 2022, Training language models to follow instructions with human feedback (arXiv:2203.02155) (behind its links)
Shumailov et al. 2023, The Curse of Recursion: Training on Generated Data Makes Models Forget (arXiv:2305.17493; later in Nature 2024) (behind its links)
Bai et al. 2022, Constitutional AI: Harmlessness from AI Feedback (arXiv:2212.08073) (behind its links)
Shanahan 2022, Talking About Large Language Models (arXiv:2212.03551) (behind its links)
Brown et al. 2020, Language Models are Few-Shot Learners (arXiv:2005.14165) (behind its links)
Hoffmann et al. 2022, Training Compute-Optimal Large Language Models (arXiv:2203.15556) (behind its links)
Radford et al. 2021, Learning Transferable Visual Models From Natural Language Supervision (CLIP) (arXiv:2103.00020) (behind its links)
Patterson et al. 2021, Carbon Emissions and Large Neural Network Training (arXiv:2104.10350) (behind its links)
Mitchell & Krakauer 2023, The debate over understanding in AI's large language models, PNAS 120(13) (doi:10.1073/pnas.2215907120; arXiv:2210.13966) (behind its links)
Sennrich, Haddow & Birch 2016, Neural Machine Translation of Rare Words with Subword Units (arXiv:1508.07909) (behind its links)
Elhage et al. 2021, A Mathematical Framework for Transformer Circuits (Transformer Circuits Thread, open write-up) (behind its links)
Geva et al. 2021, Transformer Feed-Forward Layers Are Key-Value Memories (arXiv:2012.14913) (behind its links)
Xi et al., The Rise and Potential of Large Language Model Based Agents: A Survey (arXiv:2309.07864v3) (behind its links)
Liu et al. 2023, Visual Instruction Tuning (LLaVA) (arXiv:2304.08485) (behind its links)
Tang et al. 2023, SALMONN: Towards Generic Hearing Abilities for Large Language Models (arXiv:2310.13289) (behind its links)
Templeton et al. 2024, Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet (Transformer Circuits Thread, open write-up) (behind its links)
Wallace et al. 2024, The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions (arXiv:2404.13208) (behind its links)
Schick et al. 2023, Toolformer: Language Models Can Teach Themselves to Use Tools (arXiv:2302.04761) (behind its links)
Ruder S (2016) An overview of gradient descent optimization algorithms. arXiv:1609.04747 (behind its links)
Computer program (encyclopedia) (behind its links)
Mikolov T, Yih W, Zweig G. Linguistic regularities in continuous space word representations. NAACL-HLT 2013:746-751 (behind its links)
Sumers, Yao, Narasimhan & Griffiths, Cognitive Architectures for Language Agents (arXiv:2309.02427v3) (behind its links)
Shehabi A, Newkirk A, Smith SJ, et al. (2024). 2024 United States Data Center Energy Usage Report. Lawrence Berkeley National Laboratory LBNL-2001637 (behind its links)
Lakretz Y, Desbordes T, Hupkes D, Dehaene S (2021) Causal Transformers perform below chance on recursive nested constructions, unlike humans. arXiv:2110.07240 (behind its links)
Butlin et al. 2023, Consciousness in Artificial Intelligence: Insights from the Science of Consciousness (arXiv:2308.08708) (behind its links)
Nanda N, Chan L, Lieberum T, Smith J, Steinhardt J (2023) Progress measures for grokking via mechanistic interpretability. ICLR 2023. arXiv:2301.05217 (behind its links)
Algorithm (encyclopedia, Wikimedia Enterprise snapshot 2026-11-01) (behind its links)
Chen M et al. 2021. Evaluating large language models trained on code. arXiv:2107.03374 (behind its links)
Serapio-García G, Safdari M, Crepy C, et al. (2023) Personality traits in large language models. arXiv:2307.00184 (preprint) (behind its links)
Binz M, Schulz E. 2023. Using cognitive psychology to understand GPT-3. PNAS. doi:10.1073/pnas.2218523120 (behind its links)
Garg S, Tsipras D, Liang P, Valiant G (2022) What can transformers learn in-context? A case study of simple function classes. NeurIPS 2022; arXiv:2208.01066 (v3) (behind its links)
Kantamneni S, Tegmark M (2025) Language Models Use Trigonometry to Do Addition. arXiv:2502.00873 (behind its links)
Becker J, Rush N, Barnes B, Rein D (2025) Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity. arXiv:2507.09089 (behind its links)
Porter B, Machery E (2024) AI-generated poetry is indistinguishable from human-written poetry and is rated more favorably. Sci Rep. doi:10.1038/s41598-024-76900-1 (behind its links)
Salvi, Horta Ribeiro, Gallotti, West 2024. On the conversational persuasiveness of large language models: a randomized controlled trial. arXiv:2403.14380 (behind its links)
Artificial intelligence (Wikipedia, via the Wikimedia Enterprise snapshot of 2026-11-01) (behind its links)
Oddi L, Scardapane S, Sugimoto T, Genovese D (2026) Authorship attribution and aesthetic evaluation of AI poetry: a case study with Haiku. arXiv:2609.15511 (behind its links)
Costello, Pennycook, Rand 2024. Durably reducing conspiracy beliefs through dialogues with AI. Science. doi:10.1126/science.adq1814 (PMID 39264999) (behind its links)
Lampinen AK, Dasgupta I et al. Language models show human-like content effects on reasoning tasks. arXiv:2207.07051v4 (behind its links)
Gholami A, Yao Z, Kim S, Hooper C, Mahoney MW, Keutzer K. AI and memory wall. arXiv:2403.14123 (2024); IEEE Micro (behind its links)
Stonebraker M, Pavlo A (2024) What Goes Around Comes Around... And Around... SIGMOD Record 53(2):21-37. doi:10.1145/3685980.3685984 (behind its links)
Li P, Yang J, Islam MA, Ren S (2023, v5 2025). Making AI Less 'Thirsty': Uncovering and Addressing the Secret Water Footprint of AI Models. arXiv:2304.03271 (behind its links)
Pearce H, Ahmad B, Tan B, Dolan-Gavitt B, Karri R. Asleep at the keyboard? Assessing the security of GitHub Copilot's code contributions. arXiv:2108.09293 (IEEE S&P 2022) (behind its links)
Mezzi E, Massacci F, Tuma K (2025) Large Language Models Are Unreliable for Cyber Threat Intelligence. arXiv:2503.23175 (behind its links)
Deng G et al. (2024). PentestGPT: Evaluating and Harnessing Large Language Models for Automated Penetration Testing. USENIX Security 24 (arXiv 2308.06782v2) (behind its links)
Henriot C. 2025. The AI-Augmented Research Process: A Historian's Perspective. arXiv:2508.01779 (preprint) (behind its links)
Schroeder DT, Kunst JR, et al. 2026. How malicious AI swarms can threaten democracy. Science (doi:10.1126/science.adz1697); author version arXiv:2506.06299v4 (behind its links)
Wendler C, Veselovsky V, Monea G, West R (2024) Do Llamas work in English? On the latent language of multilingual transformers. arXiv:2402.10588 (behind its links)
Bender EM, Koller A. Climbing towards NLU: on meaning, form, and understanding in the age of data. ACL 2020:5185-5198. doi:10.18653/v1/2020.acl-main.463 (behind its links)
Soviany, Ionescu, Rota & Sebe 2021, Curriculum Learning: A Survey, arXiv:2101.10382 (behind its links)
Salazar Alva RG, Nuñez M, López C, Martín Arista J 2025. AI-Driven Generation of Old English: A Framework for Low-Resource Languages (arXiv:2507.20111) (behind its links)
Computer science (Wikipedia, via Wikimedia Enterprise snapshot 2026-11-01) (behind its links)
Avigad, Mathematics and the formal turn (arXiv 2311.00007v2, 2023) (behind its links)
Romera-Paredes et al.: Mathematical discoveries from program search with large language models (Nature 2024, doi:10.1038/s41586-023-06924-6, PMC10794145, CC BY) (behind its links)
Huh, M., Cheung, B., Wang, T., Isola, P. 2024. The Platonic Representation Hypothesis. arXiv 2405.07987 (behind its links)
Lin S, Hilton J, Evans O (2021) TruthfulQA: measuring how models mimic human falsehoods. arXiv:2109.07958 (preprint) (behind its links)
Lewis et al. 2020, Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (arXiv:2005.11401) (behind its links)
Holtzman et al. 2019, The Curious Case of Neural Text Degeneration (arXiv:1904.09751) (behind its links)
Gao et al. 2022, PAL: Program-aided Language Models (arXiv:2211.10435) (behind its links)
Rubenstein et al. 2023, AudioPaLM: A Large Language Model That Can Speak and Listen (arXiv:2306.12925) (behind its links)
Liu et al. 2023, Lost in the Middle: How Language Models Use Long Contexts (arXiv:2307.03172) (behind its links)
Bahdanau, Cho & Bengio 2014, Neural Machine Translation by Jointly Learning to Align and Translate (arXiv:1409.0473) (behind its links)