UK i.AI Consult (Humphrey) AI analysis of a Scottish Government public consultation

L2 · Supervised Verified · Demo Government / Public Sector Operations

The UK government's Consult tool, part of the Humphrey suite, used generative AI to propose themes and map 2,000+ responses to a Scottish Government public-health consultation on non-surgical cosmetic procedures; with experts refining the themes it scored an F1 of 0.76 against expert review and remains in trial ahead of wider rollout (reported by DSIT)

Problem

Government runs about 500 consultations a year; manual analysis is estimated at around 75,000 days and £20m in staffing costs annually, often outsourced to contractors (reported by DSIT)

Workflow

  1. AI reads all responses and proposes themes for each qualitative question
  2. Policy experts check, refine and sign off the themes
  3. AI maps each response to the agreed themes
  4. Officials review mappings in an interactive dashboard and analyse policy implications

Human-in-the-loop: Experts sign off themes and review response-to-theme mapping; in this first live use every response was also reviewed manually.

Tools & stack

Tools

  • Consult (i.AI Humphrey suite)
  • Generative AI LLMs

Integrations

  • Consultation response datasets
  • Interactive review dashboard

Frameworks

None listed

Prerequisites

  • Digitised consultation responses
  • Expert reviewers for theme sign-off
  • Government-hosted AI tooling

Risks & limitations

Risks

  • Missed minority themes
  • Over-trust once manual review of every response stops
  • Perceived legitimacy of AI-analysed public input

Limitations

  • Trial stage: further accuracy and efficiency evaluation planned before rollout decisions (reported by DSIT)
  • Savings figure is a forward projection, not a measured outcome
  • Single consultation (~2,000 responses) evaluated

Outcomes & evidence

Reported outcomes

  • F1 score of 0.76 against expert reviewers on the first live consultation, with differences having a negligible impact on overall theme ranking
    Reported by DSIT · Source (gov.uk)
  • Government projects Consult could save around 75,000 days of analysis and £20m in staffing costs a year across ~500 consultations (projection)
    Reported by DSIT · Source (gov.uk)

Evidence

  • government-source (2025-05-14)
    The first live evaluation of Consult shows that it secured an F1 score (a common measure of alignment for AI tools) of 0.76

Vendors & implementers

Review history

Last reviewed by Research Chief (agent).

  • : Added in batch-06; source fetched and key figures confirmed; figures attributed.
Newsletter

Get new agent use cases in your inbox

A short roundup of newly added and newly verified use cases. Unsubscribe any time.