What a Police AI Trial Measured, and What It Missed

Key points

• Tasmania Police’s evaluation of Axon’s Draft One, released under Right to Information (RTI 557/26, 112 pages), measured drafting time to the minute against a baseline and a cost model but assessed harm with a single yes/no question; in Phase One, 21 of 148 AI-drafted summaries (14 per cent) carried an error or omission the reviewing sergeant judged critical to decision-making.
• The error scheme classified 12 of those 21 cases and has no category for an assertion the recording does not support, the failure the ABC reported on 23 October 2026; the composed workflow’s safety-critical error rate was measured once, at 6 per cent across 51 reports.
• Published research shows why that matters: a deployed police-policy system’s recall fell from 0.795 on isolated pairs to 0.472 across a whole library (arXiv:2610.29338), and automatic graders are weakest at spotting unsupported claims (arXiv:2610.10971), so the piece sets out six fixes, starting with a human-only baseline and a severity-weighted error taxonomy.

The most instructive public document in government AI this month is a bundle of redacted minutes from a small Australian police force. On 23 October 2026 the ABC reported that Tasmania Police had trialled Axon’s Draft One, which transcribes body-worn camera audio and drafts an incident narrative for an officer to check. The papers behind the story are public: the force’s Right to Information disclosure log carries RTI 557/26, a 112-page bundle holding the December 2024 terms of reference, status reports and the final evaluation report. The evaluating body, unnamed by the ABC, is the force’s own Business Improvement Unit. Axon was approached by the ABC and is not reported as having commented.

Read as a method, the bundle shows an asymmetry common to public-sector AI pilots. Time was measured to the minute, with a proper baseline and a cost model behind it. Harm was measured with a single yes/no question.

Tasmania is not an outlier. The UK National Audit Office’s report on DWP’s approach to innovation (HC 616 2026-27, 21 October 2026) credits a department serving about 23 million people, and spending £309.5bn on welfare in 2025-26, with test-and-learn piloting, then finds “scope for DWP to strengthen its approach further by applying good practice more consistently”. The US Government Accountability Office’s GAO-27-107837 of 14 October 2026 found the Pentagon’s Responsible AI Working Council did not meet from late 2024 until August 2025, still has no charter, and had three of ten authorised posts filled in its Responsible AI Division in February 2026. Evaluation capacity is thin everywhere. What follows is a design that spends it well.

What the Tasmanian evaluation got right, and should be copied

Phase One ran six weeks, from 29 January to 14 March 2025: six family-violence sergeants, as subject-matter experts, assessed 148 Draft One summaries generated from randomly selected historic body-worn camera recordings across all three districts, each against the original human report. The report lists its limitations frankly: officers had not been trained to film for AI, and no attending officer had corrected the mock-ups.

The second phase was split, and this is the part worth stealing. Phase 2A was nothing but baseline: for 24 weeks, from 12 March to 27 August 2025, Launceston officers timed their own existing work — 72 returns from report authors, 126 from validating supervisors. That produced 62 minutes of average entry time and 17 of validation, 79 combined, and a costed denominator: 9,712 family-violence reports a year statewide, 7.3 full-time equivalents, A$910,933. Only then, from 6 August to 30 November 2025, did Phase 2B put the tool in front of about 85 members and measure again: 31 minutes of entry, 11 of validation, 42 combined. The claimed gain — 38 minutes a report (37 by its own averages), roughly 50 per cent, more than 161 weeks and A$434,187 of wages a year — rests on a human-only measurement taken before the tool arrived, not on the tool’s own estimate of what it saved. The UK Government’s AI Playbook calls this establishing a baseline of metrics; it is the commonest thing missing from pilots reporting a percentage saving. As AI & GPP has argued about what an AI register should record, a deployment claim without a denominator is not a measurement.

One caveat travels with the 42 minutes: the figures are self-timed by volunteers. Twenty per cent of trained testers never used the tool and 26 per cent previewed without exporting a draft, so the saving is measured on the 54 per cent who did.

Where the measurement of harm stops

In Phase One, 21 of the 148 summaries — 14 per cent — contained an error or omission the reviewing sergeant judged “critical to decision making”. The report classifies some: names or roles of parties wrong in 6 cases (4 per cent); a threat, assault or risk factor missed in 6 cases (4 per cent). The other way round, in 10 cases (7 per cent) the tool described a threat, assault or risk factor the human report had not; reviewers verified those and acted on them. This is not a story about a bad tool.

But 21 critical cases were identified and 12 classified. The rest fall outside the taxonomy. So does the failure the ABC led on, recorded in a tester’s comment in the bundle: a draft stated that a complainant had made a serious allegation about prior threats when, on the officer’s review of the recording, she had said the opposite while another person spoke over her. That is neither an entity error nor an omission but an assertion the source does not support, and the scheme has no box for it. In the live phase the equivalent question returned 6 per cent. Both numbers are unweighted: a wrong surname and a fabricated allegation count the same.

Two further details are instructive. The free-text explanations of those errors were summarised by Copilot, as the report states — an AI summarising the error evidence in an AI evaluation. And the evaluation’s two instruments disagreed: 20 per cent of frontline respondents thought the safety of involved parties had decreased, which the report records as at odds with the experts’ independent assessment of the same 51 reports.

Speed is measured in minutes. A missed risk factor is measured only if somebody is paid to go looking for it.

Why component accuracy is not end-to-end accuracy

The most useful number here was published two days before the ABC story. In Policy Central (arXiv:2610.29338, 21 October 2026), Holly Baker, Michael Kuo and Kyana Pollas describe a deployed system that finds conflicts and redundancies between sections of police policy. Their benchmark holds 302 section pairs from one department’s library — some 84 policy documents, nearly 20,000 sections; handed a pair, Screen-Confirm reaches an aggregate recall of 0.795, rising to 0.885 on Conflict. Composed with retrieval, across the whole library, recall falls to 0.472 (0.615 on Conflict); Retrieve-Judge reaches 0.457.

The gap between 0.795 and 0.472 is the gap most pilot evaluations never report. In drafting, the component is the model’s accuracy on a draft a reviewer is already reading; the composed system is model plus officer plus supervisor plus specialist review. Tasmania’s safeguard is that chain, and the evaluation asserts it catches what the model misses — but the composed quantity, how often a safety-critical error survives the whole workflow, was measured once: that 6 per cent, from 51 reports. One number carrying a whole safety case is familiar; AI & GPP examined another instance in the 0.64 setting on facial-recognition thresholds.

If a model grades the drafts, grade the judge first

Many evaluations now use a model to score outputs. Louise Yorke’s study of the Arena-Hard judge (arXiv:2610.30512) ran GPT-4.1 at temperature zero on a pinned version over 500 prompts: rerunning the default setup unchanged shifted the aggregate by roughly a point while flipping the win/tie/loss verdict on 21.2 per cent of prompts; a variant that drops the reasoning requirement and other instructions changed outcomes on 28.2 percentage points more of them than unchanged reruns did, and cut the aggregate by 4.09 points. Asal Mehrabi, Mohammad Aliannejadi and Clemencia Siro, across nine prompting strategies, four judges, 27 languages, four tasks and 13 datasets (arXiv:2610.31172, over 1m pointwise scores), find that strategies do not transfer, that the judge matters more than the prompt, and that stability degrades in low-resource languages in a way aggregate reporting conceals. For a force taking statements through interpreters that is operational — and why the prompt belongs under change control, as AI & GPP has argued: a prompt is a policy.

Worse, automatic graders are weakest at the Tasmanian failure mode. In PARCEL (arXiv:2610.10971), M. Mikail Demir and M. Abdullah Canbaz label 3,396 parenthetical claims from New York State Court of Appeals decisions Supported, Refuted or Not Found: accuracy runs as high as 0.97, yet the best models still mark unsupported claims as supported with the full opinion in front of them; absence of support is harder to spot than outright contradiction, and fabricated citations that look plausible cost the most. A grader that cannot detect absence of support should not be the instrument measuring fabrication.

Six things to fix before the next pilot starts

1. A baseline that is not the tool’s output. Human-only work on matched cases, collected before deployment, as Phase 2A did. Report inter-rater agreement among the reviewers scoring quality, so a one-point difference on a ten-point scale can be told from rater noise.

2. A severity-weighted error taxonomy. Four classes at minimum, scored separately and never collapsed into one accuracy figure. DSIT’s AI Risk Management Toolkit guidance of 8 September 2026 scores likelihood and impact 1–5, with bands from Rare (under 5 per cent) to Almost Certain (over 80 per cent); the same arithmetic works here.

Illustrative figures — not a real evaluation; rates invented to show the shape of the output.

A single aggregate would make the fourth row dominate and hide the first.

3. A safety-critical miss rate reported alongside time saved, never netted against it. Report it twice: at the model’s output, and end-to-end after every review stage. The Policy Central numbers are the argument — 0.795 and 0.472 are different quantities, and only the second describes the service.

4. A judge-stability check. If a model scores the drafts, pin the version, fix the temperature, rerun every configuration unchanged at least three times, and publish prompt-level agreement alongside the aggregate, plus results by case complexity and language. Treat a verdict-flip rate near Arena-Hard’s 21.2 per cent as disqualifying for individual-case decisions.

5. A decision log. Record, per draft, what the reviewing officer kept, changed or rejected, with a time stamp. Automation bias then becomes measurable — the share of drafts accepted unedited, by case complexity — rather than assumed.

6. A stopping rule agreed in writing before the pilot starts, in the terms above: pause if the end-to-end safety-critical rate exceeds the pre-registered threshold in any two consecutive four-week windows. DSIT’s guidance says there is no final version to validate against, and that tests should be run by people independent of the project team; a stopping rule gives that independence teeth.

Tasmania’s evaluators recommended phased deployment only once the force has clear policies on ethical AI use, with training and funded support, and found no legislative amendment is required, only changes to the body-worn camera policy and the police manual. Ninety per cent of the expert reviewers recommended implementation. The force told the ABC that future use remains under review; Tom Sulston of Digital Rights Watch called the weak performance in complex cases “really worrying”.

Three things to watch: whether Tasmania Police publishes a decision, and whether the policy-first condition is honoured or inverted; whether other Australian forces trialling AI report writing publish evaluations at all; and whether any reports an end-to-end safety-critical miss rate rather than a minutes-saved figure. The first to publish both numbers will have done more for the field than any vendor benchmark.

Sources

1. AI tool from US manufacturer Axon drafts family violence reports for Tasmania Police — ABC News, 23 October 2026
2. Right to Information disclosure log — Tasmania Police, accessed 25 October 2026
3. RTI 557/26 in-scope documents (with redactions) — Department of Police, Fire and Emergency Management, Tasmania, disclosure dated 6 October 2026 (PDF; 112 pages; scanned)
4. DWP’s approach to innovation, HC 616 2026-27 — National Audit Office, 21 October 2026
5. Artificial Intelligence: Actions and Opportunities to Implement Responsible Use in DOD, GAO-27-107837 — US Government Accountability Office, 14 October 2026
6. Policy Central: Detecting Conflicts, Redundancy, and Similarity in Deployed Police Policy, arXiv:2610.29338 — Holly Baker, Michael Kuo and Kyana Pollas, 21 October 2026
7. Behind Aggregate Scores: What Systematically Varying Arena-Hard’s LLM Judge Reveals, arXiv:2610.30512 — Louise Yorke, 21 October 2026
8. Prompting Strategy Effects Depend on the Judge: A Multilingual Study of LLM-as-a-Judge Robustness, arXiv:2610.31172 — Asal Mehrabi, Mohammad Aliannejadi and Clemencia Siro, 22 October 2026
9. When Citations Mislead? A Claim-Level Benchmark for Legal Hallucination Detection, arXiv:2610.10971 — M. Mikail Demir and M. Abdullah Canbaz, 7 October 2026 (published in ICAIL 2026; DOI 10.1145/3836937.3836948)
10. AI Risk Management Toolkit: guidance — Department for Science, Innovation and Technology, GOV.UK, published 8 September 2026, last updated 19 October 2026
11. Artificial Intelligence Playbook for the UK Government (HTML) — Government Digital Service, GOV.UK, 10 February 2025

Claim ledger (piece 16)