Rejected by an Algorithm

In the first three months of Medicare’s AI-assisted prior-authorisation pilot, two contracted vendors denied 5,944 requests for care. One request waited 83 days for a response against a 72-hour standard. And under the terms of the arrangement, the vendors are paid more when they deny.

Those findings come from about 1,000 pages of records obtained by the Electronic Frontier Foundation (EFF) through Freedom of Information Act litigation and published on 8 September. On 24 September the Medicare Rights Center, a patient-advocacy organisation, published its own reading of the records, describing the model as causing inappropriate denials of care. Senator Cantwell’s office has also published a snapshot report on the programme. The pilot, known as WISeR, covers New Jersey, Ohio, Oklahoma, Texas, Arizona and Washington.

WISeR is not the most technically sophisticated system in government. It may, however, be the most instructive. It shows how algorithmic decision-making in public benefits can fail for reasons that have little to do with the algorithm.

The incentive is in the contract

Prior authorisation exists to stop unnecessary or inappropriate care being paid for. Reasonable people can support that aim, and supporters of the pilot would argue that automation can make review faster and more consistent than a paper-based process. That argument deserves to be tested, not dismissed.

But a review system is only as neutral as the incentives around it. When a vendor’s revenue rises with each denial, the design of the contract pushes in one direction regardless of how well the model performs. Any classifier makes errors in both directions, approving what should be refused and refusing what should be approved. A payment structure that rewards denials tilts every borderline call, every threshold setting and every staffing decision about human reviewers towards the cheaper error for the vendor, which is the costlier one for the patient.

When a vendor is paid more for saying no, the algorithm is not the decision-maker; the contract is.

This is a procurement problem before it is an AI problem. The same incentive would distort a team of human reviewers. Automation simply lets it operate at greater scale and with less visibility.

Timeliness is part of the decision

The 83-day case matters as much as the denial count. In health care, delay is a decision. A response that arrives weeks after a 72-hour standard may be formally an approval yet practically a refusal, because the treatment window has passed or the patient has paid out of pocket or gone without.

Automated systems are often sold on speed. If records show a system missing its own timeliness standard by a factor of more than 25, the oversight question is whether anyone was measuring, and what consequence followed. A standard without a penalty is an aspiration.

Two policy contrasts

Two recent American laws, both outside health care, illustrate safeguards that WISeR’s critics say are missing.

California’s human-review principle. On 30 September Governor Gavin Newsom signed the “No Robo Bosses Act”, reported as SB 947. It bars employers from firing or disciplining workers on automated output alone and requires human review and notice, from 1 July 2027. The principle is simple: an automated system may inform a consequential decision about a person, but may not make it unaided. Applied to benefits, it would mean every algorithmic denial is reviewed by a qualified human before it takes effect.

Colorado’s notice, correction and human-review model. Colorado’s Automated Decision-Making Technology Act (SB 26-189), signed on 14 May and effective 1 January 2027, replaced the state’s broader algorithmic-discrimination law after a federal and industry challenge. It is narrower than its predecessor, but it centres on rights that map directly on to benefits: notice that an automated system was involved, an opportunity to correct inaccurate data, and access to human review.

Neither law applies to Medicare, a federal programme. But together they describe a floor that a growing number of US legislators, across different political settings, appear to accept: people should know when a machine was involved, be able to fix the facts, and reach a human.

What oversight bodies should ask

For inspectors general, legislative committees, auditors and their counterparts in other countries considering similar pilots, the WISeR records suggest a practical checklist.

• Payment structure. How are vendors paid? Does any element of compensation rise with denials or fall with approvals? If so, what countervailing controls exist?
• Error rates in both directions. What share of denials are overturned on appeal or on human review? Overturn rates are the most direct public evidence of wrongful refusals.
• Timeliness. What proportion of decisions meet the statutory or contractual standard, what is the longest wait, and what penalties apply for breaches?
• Human involvement. Is every denial reviewed by a qualified clinician before it is issued, or only those that are appealed?
• Notice and appeal. Are patients told that an automated system was involved, given the reasons for denial, and offered a route to appeal that is realistic for elderly and unwell people?
• Transparency. Why did it take FOIA litigation for these records to become public? Pilot evaluations of this kind should be published by default.
• Expansion criteria. What evidence would justify extending the model, and who signs off?

The last question is not hypothetical. Planning documents reportedly consider expanding WISeR to air ambulances and cancer treatment, areas in which delay can be irreversible.

A lesson beyond Medicare

Governments everywhere are under pressure to control costs and modernise administration, and automated review will be part of the answer. The WISeR records do not show that automation in benefits is inherently unjust. They show that when the commercial incentive, the timeliness standard and the right of appeal are left weak, an algorithm will faithfully execute the weakness at scale.

The fix starts upstream of the model: in the contract, the service standard and the appeal route. Those are decisions made by officials, and they remain answerable for them.

Sources

1. EFF — New records reveal problems with Medicare’s AI prior-authorisation experiment
2. Medicare Rights Center — New records show Medicare WISeR model causing inappropriate denials of care
3. Office of Senator Cantwell — WISeR snapshot report
4. The Next Web — No Robo Bosses Act and Newsom’s AI bills
5. SHRM — Newsom signs revamped No Robo Bosses Act into law
6. Skadden — Colorado repeals and replaces its AI Act
7. Axios — Justice Department joins xAI challenge to Colorado AI law

Discussion

No comments yet. Start the conversation.

Discussion is open to members.

Join free Sign in