EVOPEDIA

← all systems · see it in The System

Database

An organized collection of data kept with a database management system (DBMS), 'the software that interacts with end users, applications, and the database itself to capture and analyze the data' (src:wp:database); the database, the DBMS and their applications together make a database system. The word is also used loosely (C-1397).

Properties published scienceour proposal

Sourced fields, and our own readings.

born at1960s: the term 'data-base' in a 1962 SDC report; disks widely available mid-1960s; IDS and IMS (1966)
born precisiondecade
boundaryInside: the stored data with its structure and semantics ('the data, the structure, and the semantics'), and the DBMS that keeps it: storage engine and indexes, transaction manager and log, query processor and optimizer, catalog and schema, access control. Outside: the hardware and operating system it runs on, the network, the applications and people who use it, the world the data model, attackers, other databases (replicas, warehouses, federations). Whether 'database' names the data, the DBMS or the whole system is contested (C-1397).
nested levelBuilt from the storage engine, indexes, log, transaction manager, optimizer, schema and language; runs on the computer's memory and storage through the operating system; carries applications, websites, warehouses and science archives that run on it.
scaletechnological

Measures (21) published science

mirrored disc pair mean time to failure (realistic analysis)800 years
characteristic clock: Dynamo service-level agreement (99.9th percentile)300 ms
characteristic clock: disc mean time to failureabout 1 year
disc repair timeabout 1 hour
unmirrored eight-disc system: unavailable discabout 8 per year
transactions that wait on a lock1 in 1000 transactions
deadlock rate scalingsquare of multiprogramming degree; fourth power of transaction size deadlocks per second
cardinality misestimates, all systems tested (per-hand-off toll on the join chain)factor of 1000 or more, routinely; errors grow exponentially with joins estimate / true size
Join Order Benchmark queries113 queries, 3 to 16 joins, average 8 joins per query
largest IMDB tables in the benchmark36 M and 15 M rows
learned index vs cache-optimized B-treeup to 70% faster; an order of magnitude less memory speed, memory
divergent versions seen per request (shopping cart, 24 h)99.94% one version; 0.00057% two; 0.00047% three; 0.00009% four share of requests
NAR Molecular Biology Database Collection size (2025)2236 databases
NAR collection turnover in one year (2025)74 added; 226 discontinued URLs removed; 932 reviewed entries
unique molecular-biology databases in NAR Database Issues 1991-20161727 (in 3055 articles) databases
new molecular biology databases per yearover 100 per year
databases still available by citation quartile (debuted before 2013)Q1 155 of 193 (80%); Q2 232 of 338 (69%); Q3 236 of 385 (61%) databases
vector search added to RDBMSs after ChatGPTless than one year
Amazon platform scaletens of millions of customers at peak; tens of thousands of servers customers, servers
industry size reached by about 1990a billion-dollar industry in about ten years US dollars
SQL standardizedANSI 1986; ISO 1987 year

Open contentions (4) open contention

Where sources or readings disagree. Both stay on the map.

Cross-scale patterns seen here (20) our proposal

Shapes that recur when the same kind of problem is solved at different scales, and how each shows in Database. Each links to every other system that shows it.

Valence: what it trades, and with whom (23) published science

Each exchange with a neighbour, read from both sides: who gains, who pays, and what carries it.

Partners and parts published science

Built from / builds (12)

Sits inside / holds (4)

expressed by (25)

floor filled by (24)

Kinds (25)

wired (2)

Notes published science

Layers

The ways it takes in its world, where they meet, and how it acts back, each from L1 (where it meets the world) up.

the boardroom (shared floors)

  1. L3: Transaction manager: atomicity, consistency, isolation, durability; locks, the log, commit and abort, recovery; replication
  2. L4: Query processor: the optimizer (cardinality estimates, cost model, join order, plan enumeration) and the executor; in self-driving DBMSs, the planning module
  3. L5: Conceptual level: the schema, keys and integrity constraints, the catalog, access control and security monitoring
  4. L6: External level: the views each group of users sees, and the languages and APIs (SQL, ODBC, JDBC) through which they reach it
  5. L7: The world the data model and the people and organizations who use it: applications, analysts, designers, administrators, scientific communities

the database as database people describe it: data coded as bits on storage up to the world the data model (reader's view)

  1. L1: Storage: the data coded into bits on disks, flash and main memory (and the stream of records arriving from the world through applications)
  2. L2: The internal (physical) level: the storage engine's pages, row or column layouts, indexes (B-trees, hashes, bitmaps), materialized views and the log on disk
  3. L3: Transactions: the transaction manager, locking and concurrency control, logging, commit and abort, crash recovery, replication (ACID)
  4. L4: Query processing: parser, the query optimizer (cardinality estimates, cost model, join order) and the executor
  5. L5: The conceptual (logical) level: the schema of relations, keys and integrity constraints, the catalog, and access control
  6. L6: The external level: views for each group of users, and the languages and APIs that reach them (SQL, ODBC, JDBC)
  7. L7: The world the data model and the people and organizations that use it: operations, reservations, inventories, libraries, science archives

the replicated database across machines: links between nodes up to reconciled versions (Dynamo-style key-value store)

  1. L1: Many commodity servers in several data centres, linked by networks; components fail all the time
  2. L2: Membership and failure detection by gossip; partitioning by consistent hashing
  3. L3: Replication with sloppy quorums and hinted handoff; anti-entropy with Merkle trees
  4. L4: Versioning and reconciliation: vector clocks, application-assisted conflict resolution; consistency traded for availability

way in: Access: credentials and privilege grants (DCL)

  1. L1: A person or program asks for access, giving a password
  2. L2: Privileges are looked up by individual, group or role

way in: Administration: the database administrator's settings and actions

  1. L1: The DBA decides: registering users, monitoring performance, tuning
  2. L2: Dedicated user interfaces for administrators
  3. L3: Configuration parameters (knobs) tuned statically or dynamically, such as main memory used

way in: Attacks: crafted input (SQL injection) arriving through an application

  1. L1: An attacker types crafted input into a web application's form
  2. L2: The web application passes it on to the backend database inside a query
  3. L4: The injected statement is executed as a query

way in: Deposits: data submitted by a community (scientific archives such as the PDB)

  1. L1: Structures measured by crystallography, NMR, electron microscopy and micro-electron diffraction in labs on every inhabited continent

way in: Queries: requests for data by content (DQL)

  1. L1: A question about the world is put as a declarative query: what data is wanted, not where it is
  2. L2: The query text is received by the language processor
  3. L3: Because queries are mathematical logic, they can be rewritten in provably correct ways

way in: Replicas: messages from other nodes (gossip, replica writes, Merkle-tree comparisons)

  1. L1: Messages from other servers over the network; routes flap and data centres fail
  2. L2: Gossip-based membership and failure detection; hinted writes held for absent replicas

way in: Schema: data definition from designers and administrators (DDL)

  1. L1: The designer's conceptual data model of the application domain (entities, relationships, attributes)
  2. L2: Data definition statements create, alter or drop tables and their relationships
  3. L3: The definitions are written into the catalog
  4. L4: Physical design: indexes and layout chosen for the expected workload, invisible to applications

way in: Self: statistics about its own data (cardinalities, histograms)

  1. L1: The stored data's own distribution: how many rows match, which values cluster together
  2. L2: Statistics gathered by a statistics tool: histograms, samples, column-group statistics
  3. L3: Cardinality estimates for each intermediate result

way in: Self: the workload over time (what queries arrive, how often)

  1. L1: The stream of queries and transactions arriving through the day
  2. L2: Queries classified (OLTP or OLAP) and clustered
  3. L3: Workload forecasting models (recurrent neural networks, LSTM) predict future demand

way in: Storage: the medium's own state (I/O completions, bad spots, failed discs, lost memory)

  1. L1: Discs and memory: a typical disc fails about once a year (bad spots, spindle, the path to the disc)
  2. L2: Fail-fast modules report: either they work or they fail and do nothing

way in: Time: the clock, the order of events, and the human day

  1. L1: Time passing in the world: users' days and nights shape the load
  2. L2: Time stamped into the data: valid-time and transaction-time in temporal databases; vector clocks for versions

way in: Writes: inserts, updates and deletes from applications and people (DML)

  1. L1: A fact in the world becomes a statement sent by an application or typed by a person: insert, update, delete
  2. L2: The statement arrives through the DBMS's API or language processor

way out: Abort and undo; reject what breaks the rules

  1. L1: The stored state is as if the transaction never ran
  2. L3: Undo each action in the transaction's log; free its locks

way out: Back up, and restore a previous state

  1. L1: The files are used to restore that state
  2. L3: Each desired database state is kept within dedicated backup files

way out: Commit: make a change durable on storage

  1. L1: Bits on durable storage
  2. L2: Log records and pages are written by the storage engine
  3. L3: The transaction commits: all its effects survive, or it aborts and none do

way out: Return results to the application and the person

  1. L1: Rows appear on a screen or flow into the application
  2. L4: The executor computes the query result along the chosen plan
  3. L6: Results are presented through the user's view, by an SQL tool or a website

way out: Self-driving actions on its own physical design

  1. L2: Actions: add or drop indexes and materialized views, row to columnar layouts, compress, repartition, move tiers, set knobs
  2. L4: The planner chooses an action sequence by estimated cost-benefit

way out: Send updates to replicas

  1. L1: Writes travel to the N hosts that hold the key; Merkle-tree comparisons run in the background
  2. L3: Updates of a replicated object are synchronized across the copies

way out: Triggers and alerts (active databases)

  1. L1: Security monitoring, alerting, statistics gathering, authorization
  2. L5: An event-driven rule fires on a condition inside or outside the database

way out: Write the audit trail

  1. L1: A record kept for a later forensic audit
  2. L5: Change and access logging records who accessed which attributes, what changed, and when

Sources (22) published science

Database (Wikipedia, via Wikimedia Enterprise snapshot 2026-11-01)
Gray J (1981) The Transaction Concept: Virtues and Limitations. Tandem Technical Report TR 81.3 (VLDB 1981) (behind its links)
Pavlo A et al. (2017) Self-Driving Database Management Systems. CIDR 2017 (behind its links)
Leis V, Gubichev A, Mirchev A, Boncz P, Kemper A, Neumann T (2015) How Good Are Query Optimizers, Really? PVLDB 9(3):204-215. doi:10.14778/2850583.2850594 (behind its links)
DeCandia G et al. (2007) Dynamo: Amazon's Highly Available Key-value Store. SOSP 2007. doi:10.1145/1294261.1294281 (behind its links)
Codd EF (1970) A Relational Model of Data for Large Shared Data Banks. Communications of the ACM 13(6):377-387. doi:10.1145/362384.362685 (behind its links)
Stonebraker M, Pavlo A (2024) What Goes Around Comes Around... And Around... SIGMOD Record 53(2):21-37. doi:10.1145/3685980.3685984 (behind its links)
Imker HJ (2018) 25 Years of Molecular Biology Databases: A Study of Proliferation, Impact, and Maintenance. Front Res Metr Anal 3:18. doi:10.3389/frma.2018.00018 (behind its links)
Rigden DJ, Fernandez XM (2025) The 2025 Nucleic Acids Research database issue and the online molecular biology database collection. Nucleic Acids Res 53(D1):D1-D9. doi:10.1093/nar/gkae1220 (behind its links)
Kraska T, Beutel A, Chi EH, Dean J, Polyzotis N (2018) The Case for Learned Index Structures. arXiv:1712.01208v3 (SIGMOD 2018) (behind its links)
Kent, K. & Souppaya, M. (2006). Guide to Computer Security Log Management. NIST Special Publication 800-92. doi:10.6028/NIST.SP.800-92 (behind its links)
Gilbert S, Lynch NA (2012) Perspectives on the CAP Theorem. IEEE Computer 45(2):30-36. doi:10.1109/MC.2011.389 (behind its links)
Kindy DA, Pathan A-SK (2012) A Detailed Survey on Various Aspects of SQL Injection in Web Applications: Vulnerabilities, Innovative Attacks, and Remedies. arXiv:1203.3324 (behind its links)
Anderson R (2008) Security Engineering, 2nd edition, chapter 4 Access Control (author's free copy) (behind its links)
Span of control (Wikipedia, via Wikimedia Enterprise snapshot) (behind its links)
Jonas E, Schleier-Smith J, Sreekanti V, et al. (2019). Cloud Programming Simplified: A Berkeley View on Serverless Computing. arXiv:1902.03383 (behind its links)
Set theory (Wikipedia, via Wikimedia Enterprise snapshot 2026-11-01) (behind its links)
Computer science (Wikipedia, via Wikimedia Enterprise snapshot 2026-11-01) (behind its links)
Data structure (encyclopedia, Wikimedia Enterprise snapshot 2026-11-01) (behind its links)
Jackson R et al. OBO Foundry in 2021: operationalizing open data principles to evaluate ontologies. Database 2021. doi:10.1093/database/baab069 (behind its links)
Sorting algorithm (Wikipedia, via Wikimedia Enterprise snapshot 2026-11-01) (behind its links)
Burley SK et al. (2022) RCSB Protein Data Bank: Celebrating 50 years of the PDB with new tools for understanding and visualizing biological macromolecules in 3D. Protein Sci 31(1):187-208. doi:10.1002/pro.4213 (behind its links)