← all systems · see it in The System
Computer ethics
The part of practical philosophy concerned with how computing professionals should make decisions about professional and social conduct (src:wp:computer-ethics); in Maner's founding definition, the study of ethical problems 'aggravated, transformed or created by computer technology' (src:bynum-sep-2015-computer-ethics). Walked as the field describes itself: the effects computing has on people, the cases and audits that bring them to notice, the policy vacuums they open (Moor), the analysis, the codes, principles and review frameworks, the shared values, and the policies, designs, teaching and laws sent back out. AI ethics, professional (software engineering) ethics and ICT research ethics are booked as its kinds.
Properties published scienceour proposal
Sourced fields, and our own readings.
| born at | 1940s roots (Wiener); named 1976 (Maner); a field by 1985 |
| born precision | decade |
| boundary | Inside: the scholars and practitioners who do computer ethics, their cases and audits, their analysis (analogies, theories, Moor's method), the professional codes and AI guidelines, the research ethics frameworks (Menlo) and review, the shared values and principles, and the policies, designs and teaching they produce. Outside: the computing technology and the people and societies it affects, the law that may or may not adopt its policies, the companies and states that build and deploy systems, and the older ethical traditions it borrows from. |
| nested level | A branch of applied (practical) ethics, like medical and business ethics (src:bynum-sep-2015-computer-ethics), its foundations investigated by information ethics (src:wp:computer-ethics); practised by people in professions, universities and review boards. |
| scale | social |
Measures (17) published science
| AI ethics guideline documents (to 2019) | 84 documents |
| share released after 2016 | 88 % |
| documents by private companies / governmental agencies | 19 (22.6%) / 18 (21.4%) documents |
| principle frequency: transparency / justice and fairness / non-maleficence / responsibility / privacy | 73 / 68 / 60 / 60 / 47 of 84 documents |
| issuers in the USA and UK | 20 (23.8%) and 14 (16.7%) documents |
| code-exposure test: participants | 105 developers (5+ years), 63 graduate students; 11 scenarios people |
| effect of seeing the ACM code on responses | no statistically significant difference for any vignette |
| commercial gender classification: worst error (darker-skinned women) / best-case group error (lighter-skinned men) | up to 34.7 / 0.8 % |
| Black patients flagged for extra help: algorithm as built / with the disparity remedied | 17.7 / 46.5 % |
| Therac-25 patients overdosed / deaths contributed to | 6 / 3 patients |
| Moral Machine decisions | 40 million, 233 countries and territories, ten languages decisions |
| Therac-25 overdose size | up to 250 times normal dose |
| NCCV 1991 attendance | 400 people, 32 US states, 7 other countries |
| characteristic clock: ACM code revisions | 1973, 1992, 2018 (19 and 26 years apart) |
| computing ethics teaching papers reviewed / mentioning assessment | 100 / 59% papers |
| characteristic clock: Belmont standards to federal regulation | over a decade years |
| naming to watershed | 1976 to 1985 years |
Open contentions (6) open contention
Where sources or readings disagree. Both stay on the map.
- C-413 Where do thoughts come from?
booked: No theory is proven, so the map lists the main ones side by side and doesn't pick one. It shows where thinking is handled in the brain, not where thoughts start.
the other reading: see C-413 - C-1690 Are AI ethics guidelines converging on shared principles?
booked: The map books convergence on five principles and keeps the view that the agreement hides deep disagreement over meaning.
the other reading: a global convergence is emerging (Jobin et al.); the consensus hides deep political and normative disagreement (Mittelstadt); the convergence resembles medicine's four principles: shared ancestry, not independent agreement - C-1700 When did computer ethics begin?
booked: The map lists Wiener's 1940s work, Maner's naming in 1976 and the 1985 watershed, ruling none.
the other reading: 1940s-1950: Wiener's Cybernetics (1948) and The Human Use of Human Beings (1950) laid the foundations, though he did not name a field; 1976: Maner coined 'computer ethics' and taught it; 1985: Johnson's textbook and Moor's essay made it a field ('watershed year') - C-1701 Does computing raise new ethical problems or old ones in new form?
booked: The map books both positions.
the other reading: Maner: computers create wholly new ethical problems that need new values and principles; Johnson: old problems 'given a new twist', new species of old moral issues; Górniak: computer ethics will become a global ethic and replace local ethics - C-1702 Can a computer be a moral agent?
booked: The map does not book computers as moral agents and lists the debate.
the other reading: most philosophers: current computers should not be called moral agents if that means holding them responsible; some authors rethink moral agency for artificial agents (Floridi and Sanders; Sullins; Moor's kinds of agents); Weizenbaum: deciding is computational, choosing is not - C-1713 Do AI ethics principles hold off real regulation?
booked: The map books the critics' reading and keeps the view that principles focus public debate.
the other reading: self-regulatory AI principles give policy-makers a reason not to regulate, serving as virtue signalling (Mittelstadt; Hagendorff; Jobin cites the critique); the principles focus public debate and raise awareness (Mittelstadt notes this too)
Partners and parts published science
Built from / builds (5)
- Computing professionals
- Professional codes of ethics (ACM 1973, 1992, 2018; IEEE; BCS; ACS; GI; LOPSA; IEEE/ACM Software Engineering Code 1999)
- AI ethics guidelines (84 documents by 2019)
- The Menlo Report and ICT research ethics review
- Scholars, journals, conferences and courses (Metaphilosophy 1985, NCCV 1991, ETHICOMP 1995)
Sits inside / holds (2)
- Human society (from local community up to societies)
- Ethics (moral philosophy: the study and the morality it studies)
Trades with (valence) (13)
- Privacy via sys:human-society · Computer ethics gain
- Ethics (moral philosophy: the study and the morality it studies) via sys:human-society · Computer ethics gain
- Penetration testing via sys:law · Computer ethics gain
- Human (Homo sapiens) via sys:human · Computer ethics gain
- Law (rules and law) via sys:law · Computer ethics gain
- Software via sys:software · Computer ethics gain
- Algorithm (the practice of step-by-step procedures and the structure they compute) via sys:algorithm · Computer ethics gain
- Computer security via sys:computer-security · Computer ethics gain
- Artificial intelligence via sys:computer · Computer ethics gain
- Organization via sys:organization · Computer ethics mixed
- Medical ethics (bioethics) via sys:ethics · Computer ethics gain
- ACM (professional society) via sys:organization · Computer ethics gain
- Computer law via sys:human-society · Computer ethics gain
expressed by (13)
- Generation/Synthesis
- Templating / Standardization
- Interoperability/Transduction (Bridging)
- Internal Coordination/Regulation
- Sensing/Scanning
- Uncertainty Reduction/Learning
- Trail-Marking / Externalized Memory (Stigmergy)
- Social Learning / Public Information Use
- Reactive Response
- Proactive Defense
- Self-Monitoring
- Validation of Effectiveness
- Quality Control & Error Correction
floor filled by (15)
- Human (Homo sapiens)
- Law (rules and law)
- Software
- Human society (from local community up to societies)
- Computer
- Algorithm (the practice of step-by-step procedures and the structure they compute)
- Computing professionals
- Professional codes of ethics (ACM 1973, 1992, 2018; IEEE; BCS; ACS; GI; LOPSA; IEEE/ACM Software Engineering Code 1999)
- AI ethics guidelines (84 documents by 2019)
- The Menlo Report and ICT research ethics review
- Scholars, journals, conferences and courses (Metaphilosophy 1985, NCCV 1991, ETHICOMP 1995)
- Value sensitive design
- Ethics (moral philosophy: the study and the morality it studies)
- ACM (professional society)
- Computer law
Kinds (9)
- Information ethics
- Ethics of artificial intelligence
- Professional (software engineering) ethics
- ICT research ethics
- Cyberethics
- Hacker ethic
- Machine ethics
- Value sensitive design
- Ethics (moral philosophy: the study and the morality it studies)
Notes published science
- Codes of ethics may not change what programmers do: when 105 experienced developers and 63 graduate students were given 11 real ethical dilemmas, half of them after reading the ACM's code of ethics, there was no statistically significant difference in any of their answers. Codes seem to work only when a workplace builds them into its culture and enforces them (src:hagendorff-2020-ethics quoting McNamara et al.; src:ncsu-2018-acm-code-study; src:mittelstadt-2019-principles).
- Computer ethics was born in a hospital ethics class: in 1976 Walter Maner noticed that his medical ethics cases got harder, or turned into new problems, once computers were involved. Fifty years later AI ethics borrowed medicine's four principles again, and a critic points out that AI development lacks what made them work in medicine: shared aims, duties to a patient, a professional history and real accountability (src:bynum-sep-2015-computer-ethics; src:mittelstadt-2019-principles).
- A health algorithm used on millions of patients was racially biased without ever looking at race: it predicted how much money patients would cost, not how sick they were, and less is spent on Black patients. Fixing it would raise the share of Black patients flagged for extra help from 17.7% to 46.5% (src:obermeyer-2019-bias).
Layers
The ways it takes in its world, where they meet, and how it acts back, each from L1 (where it meets the world) up.
computer ethics as its scholars, codes and review boards describe it: computing's effects on people up to the policies and designs that answer them (reader's view)
- L1: Computing at work in the world: what computers and their makers do to and for people
- L2: Noticing: accident reports, audits, surveys, scholars' and practitioners' observations
- L3: Naming the problem: policy vacuums and conceptual muddles
- L4: Ethical analysis
- L5: Codes, principles and review
- L6: Shared values and responsibility
- L7: Policies, designs, teaching and law sent back out
the boardroom (shared floors)
- L4: Ethical analysis: muddles cleared, analogies tested, theories applied (consequentialism, deontology, virtue, Moor's just consequentialism, Wiener's method, Floridi's information ethics)
- L5: Codes, principles and review: professional codes (ACM 1973, 1992, 2018; IEEE/ACM Software Engineering Code 1999), AI guidelines, research ethics frameworks (Menlo 2012) and review boards
- L6: Shared values and responsibility: core human values, the converging principles (transparency, justice and fairness, non-maleficence, responsibility, privacy), who answers for harm
- L7: Action back on the world: policies formulated and justified, designs changed, people taught, laws proposed
way in: Accidents and failures of computer systems
- L1: The accident or harm in the world: Therac-25's overdoses (1985-87), the 737 MAX crashes, the USS Vincennes shoot-down; ICT research that harmed people
- L2: Investigation: accident reports and analyses (Leveson and Turner on Therac-25; the 737 MAX investigations)
- L3: Causes traced across many hands: software errors, poor testing, overconfidence, bad interfaces, dismissed reports; no single person to blame
way in: Audits of what algorithms decide about people
- L1: Automated decisions about people: hiring, loans, sentencing risk scores, health-care management, face analysis
- L2: Audits: balanced benchmarks (the Pilot Parliaments Benchmark, 1270 people) and outcome data compared across groups
- L3: Disparities measured and traced to a cause: error rates by skin type and gender; cost used as a proxy for illness
way in: Ethical traditions and borrowed frameworks
- L1: The received policy cluster: a society's existing laws, rules, practices; ethical theories
- L2: Analogies: conceptual bridges to past cases, carrying intuitions across
- L3: Borrowed frameworks: medical ethics' four principles (principlism) taken up by AI ethics; the Belmont principles restated for ICT research
way in: Guidelines issued by companies, governments and others
- L1: Issuers and their priorities: private companies (19 documents), governmental agencies (18), academic bodies (9) and others
- L2: Expert committees and corporate teams draft principles
- L3: The published corpus: 84 documents, 88% after 2016
- L4: Meta-analyses map the corpus: eleven principles, overlaps and omissions
way in: Law and regulation
- L1: Statutes and regulations: the Right to Financial Privacy Act (1978), the Computer Matching and Privacy Protection Act (1988), 45 CFR 46 (the Common Rule)
- L2: Legal requirements reach researchers and practitioners
- L3: Discordant legal regimes across borders: whose law applies on the Internet
way in: New capabilities computing opens
- L1: Logically malleable machines: any activity describable as inputs, outputs and logical operations
- L2: New choices for action noticed by practitioners, scholars and the public
- L3: A policy vacuum named, often with a conceptual muddle
way in: Public moral preferences
- L1: People's moral intuitions about machine decisions, across countries
- L2: Collected by an online platform (Moral Machine, January 2016 to July 2020): 40 million decisions, 233 countries and territories
- L3: Global preferences, individual variation and three clusters of countries
way in: The practitioner's own moral sense
- L1: The individual's own personal ethical code
- L2: Moral intuitions transferred across an analogy to the new case
- L3: Moral imagination brought to design (value sensitive design)
way in: The scholarly conversation
- L1: Scholars from many disciplines: computer scientists, philosophers, policy makers, lawyers, journalists, sociologists, psychologists
- L2: Journals, special issues, conferences (Metaphilosophy 1985, NCCV 1991, ETHICOMP 1995)
- L3: Debates: the uniqueness debate, the Górniak hypothesis, agent- and patient-based theories
way in: The workplace and its codes
- L1: Day-to-day work decisions of computing professionals
- L2: Any informal code of ethical conduct in the workplace
- L3: Exposure to formal codes of ethics
- L4: Codes take effect only when embedded in organisational culture and enforced
way in: What the classroom shows
- L1: Students in computing courses, stand-alone or with ethics embedded in technical courses
- L2: Assessments, self-report surveys, course evaluations, instructor reflections
- L3: Published accounts of what worked (100 papers through 2022)
way out: Codes published to practitioners
- L1: The practitioner's work decisions
- L2: Read by the practitioner (no measurable change in choices in one test)
- L3: Published to members, shown in courses and studies
- L5: A code drafted and adopted (ACM 1992: 24 statements of personal responsibility; updated July 2018)
way out: Design changed (value sensitive design, privacy by design)
- L1: The artefact built to prevent the harm from the start
- L4: Conceptual, empirical and technological investigations, iterated
- L7: Values of direct and indirect stakeholders chosen
way out: Policies and law proposals
- L1: Conduct governed: what may be done with the technology
- L5: Taken up as law in some cases (Right to Financial Privacy Act 1978)
- L7: Policy formulated to fill the vacuum, then justified
way out: Published critique and audits
- L1: Public, press and regulators informed
- L6: Findings framed as harms to shared values
way out: Research ethics review
- L1: Research run, changed or stopped
- L3: Stakeholder analysis and ethical impact assessment; review boards
- L5: A framework: respect for persons, beneficence, justice, respect for law and public interest
way out: Teaching
- L1: Students who will build computer artefacts
- L3: Courses: stand-alone, or ethics embedded in technical courses
- L5: Curricula: Maner's starter kit (1978, published 1980), ABET accreditation, the ACM/IEEE-CS/AAAI curricula