An algorithm used by the federal government to assess the level of care older Australians are entitled to is unreasonable, unfair and lacks human oversight, according to a damning investigation by the commonwealth ombudsman.
The investigation also found transparency recommendations from the robodebt royal commission have been ignored.
The legislation must be changed to allow aged care assessors to change clinically incorrect algorithmic decisions themselves, and there are also significant legal issues with the tool, the commonwealth ombudsman, Iain Anderson, told Guardian Australia on Monday.
The investigation followed reporting by Guardian Australia that older people were being left distressed and without adequate care after their home support needs were assessed by an “inhumane” algorithm as part of the government’s mandatory Integrated Assessment Tool (IAT).
Aged care assessors are barred from overriding the IAT’s funding package recommendation, even when the decision is wrong.
“Under the law, these automated decisions cannot be overridden and changed, even if they are clinically inappropriate,” the ۱۰۴-page ombudsman report found.
“There is no public information available that meaningfully explains how the algorithm converts inputs into outcomes. Without this, the IAT is a technological black box.”
It meant the IAT and its algorithm “do not merely have the potential to undermine public confidence in the administration of a fair and consistent assessment system,” the report found. “Rather, a reasonable person would, in our view, lack confidence that the system is in fact fair and consistent.”
Guardian Australia reported that those involved in developing the IAT had no idea that an algorithm would be added to it by the government by the time changes were rolled out in November.
Documents released under freedom of information reveal departmental panic, severe clinical distress among frontline assessors, and secret IT workarounds after officials realised in the hours before the rollout of the algorithm that legislation legally barred human override, and that this was causing distress within days of implementation.
“It’s very concerning that the department was telling people this system was going to operate in a particular way, showing them versions of that system, only to change it,” Anderson told Guardian Australia.
The report found: “Information provided by the department to the public suggests that key trials and consultation over several years directly supported the IAT and its associated decision-making framework in its current form, but that is not correct”.
“Those trials and consultation related to earlier versions of the IAT and earlier legislation. There was no meaningful consultation.”
Since the rollout there had been a failure of oversight, the investigation found. The qualifications and skill of assessors are critical to the IAT, but the Ombudsman found the department failed to enforce contractual quality controls or verify whether assessors held mandatory clinical qualifications.
Senior staff raised concerns about a lack of review pathways for wrong decisions before the rollout of the algorithm, the Ombudsman found. One email read; “I am concerned that our current framework does not satisfy the first of these requirements (that there be a clear path for those affected to seek review), because it appears [that] the review pathways only allow for the algorithm outcomes to be confirmed”.
after newsletter promotion
Anderson said a range of concerns had been raised by the media and public about the operation of the IAT for support at home packages, triggering the investigation in April, and in particular there was a lack of transparency around how the IAT algorithm worked in making its decisions.
“The department has not complied with recommendations … of the royal commission into the robodebt scheme regarding transparency of government automated decision-making,” the report found.
The report calls for a comprehensive review of the legislation related to the needs assessment process, and of the rules underpinning the algorithm.
“The robodebt royal commission said that when government uses automated decision making or artificial intelligence, it needs to be very transparent so people can understand how are decisions being made that are impacting them significantly,” he said.
“The department needs to much more effectively, in our view, give oversight to the assessment providers because the inputs that are provided by assessment providers to the algorithm are really critical in determining what then comes out of the other end through the algorithm.”
While the department of health and ageing in July announced a new review process, where aged care assessors can escalate wrong decisions to the department for review, Anderson said he is concerned this is not adequate.
“I think that assessors being able to change decisions because they appear to be clinically inappropriate is really important, and so I am concerned that the escalation pathway will not necessarily do that,” he said.
Anderson said numerous legal concerns had been identified by the investigation that it is waiting for the government to address.
“How can you actually challenge decisions when you don’t know how they’re made by the algorithm?” he said.
“There are significant legal issues, and we think that those need to be addressed as a matter of urgency. In our view, the department did not, and may still not, understand key components of the legal framework it created, nor did it understand what legal framework it was meant to create.”
The department of health and ageing has been contacted for comment.
