5.2.2 AI, machine learning and robotics

Ethical and Legal Issues in AI and Automation

Artificial intelligence (AI), machine learning (ML) and robotics are transforming how decisions are made and tasks are performed. Their increasing use raises significant ethical and legal questions. Four key issues are covered at GCSE: accountability, safety, algorithmic bias and legal liability.

Accountability asks: who is responsible when an AI system makes a decision that harms someone?

Traditional decisions have a clear human decision-maker who can be held responsible. AI systems make decisions autonomously — often in ways that are difficult to explain, even to their creators. This creates an "accountability gap":

  • A loan application rejected by an AI system — who is accountable? The developer? The deploying company? The training data?
  • A self-driving car involved in a collision — the manufacturer, the software team, or the passenger who chose to use it?
  • An AI system used in criminal sentencing — if it gives a harsher sentence to one group, who bears responsibility?

Emerging frameworks require AI systems to be explainable ("explainable AI") so that affected individuals can understand and challenge decisions.

Safety concerns the risk of harm when AI and robotic systems fail or behave unexpectedly. Unlike traditional software bugs, AI failures can be subtle and difficult to predict:

  • Medical AI: a diagnostic AI that misses a cancer diagnosis or recommends a harmful treatment has direct life-or-death consequences.
  • Autonomous vehicles: a self-driving car that misjudges a pedestrian's trajectory could cause fatal accidents.
  • Industrial robots: a malfunctioning robot arm in a factory could injure workers.
  • AI in weapons systems: autonomous weapons that target without human authorisation raise acute safety and ethical concerns.

Safety requires extensive testing, fail-safe mechanisms, human oversight at critical decision points, and clear standards for when AI is and is not appropriate to use autonomously.

Algorithmic bias occurs when an AI system systematically produces unfair outcomes for certain groups. It arises from biased training data or flawed design:

  • Training data bias: if historical data reflects past discrimination (e.g. fewer women in senior roles), an AI trained on it will perpetuate and amplify that discrimination — rejecting women's job applications even when equally qualified.
  • Facial recognition: systems trained predominantly on lighter-skinned faces have higher error rates for darker-skinned individuals, leading to wrongful identification.
  • Credit scoring: AI models may penalise protected characteristics (e.g. postcode as a proxy for race) without any human reviewing the basis for decisions.

Algorithmic bias is particularly insidious because it can appear objective (it is "just maths") while encoding discrimination at scale, affecting millions of decisions simultaneously.

Legal liability concerns who is legally responsible for harm caused by AI systems. Current legal frameworks were designed for human decision-makers and do not map cleanly onto autonomous systems:

  • If an AI system provides faulty medical advice and a patient is harmed, can the developer be sued for negligence?
  • Should AI systems have some form of legal personhood (like corporations) so they can be held liable?
  • How does product liability law apply to AI decisions as opposed to physical products?

The UK and EU are developing AI-specific legislation (e.g. the EU AI Act) to assign liability more clearly. At present, liability typically falls on the deploying organisation, but this area remains contested and rapidly evolving.

 Key Takeaways

  • Accountability: it is unclear who is responsible when an AI system causes harm — the developer, company or algorithm itself.
  • Safety: AI failures in high-stakes domains (medical, automotive, military) can have severe consequences and require human oversight.
  • Algorithmic bias: AI systems trained on biased historical data can perpetuate and amplify discrimination at scale.
  • Legal liability: existing law struggles to assign responsibility for harm caused by autonomous systems; new frameworks are being developed.