Governance · Algorithmic accountability
Companies without a human signature: when no one answers for the decision
Several automated systems caused real harm without any person able to explain — or own — the decision. The pattern repeats and has a technical name.
There is a question that every organization with automated systems should be able to answer in seconds: when this decision goes wrong, who explains it and who answers for it? When the answer is silence, the problem is already in place, even if it has not yet erupted.
The public cases of the last decade show the same mechanism again and again. They did not fail from an excess of technology; they failed from the absence of a human point of control.
Four failures, one pattern
A state program for automated recovery of welfare debt issued mass claims calculated by an algorithm, without case-by-case human review; an inquiry commission declared it unlawful after the harm was done. A welfare fraud detection system was struck down by a court for operating without transparency or proportionality over entire populations. A conversational agent without safety guardrails sustained interactions that no competent person would have allowed. An impersonation fraud using synthetic video — a deepfake — induced an employee to transfer tens of millions of dollars.
Four different contexts, one same signature: the decision was executed without a person with the competence, authority, and obligation to answer for it.
The crisis of the human signature
The problem is not artificial intelligence. It is the disappearance of the place where someone signs. In a healthy system, there is a point where a person reviews, decides, and becomes associated with that decision: if it goes wrong, it is known whom to ask. Automation without governance dissolves that point and spreads it across a model, a vendor, and a process that no one fully owns.
Automation does not eliminate responsibility: it makes it invisible until someone demands it.
What control would have prevented it
None of these cases required halting the technology. It required three things that an AI management system under ISO/IEC 42001 obliges an organization to document: meaningful human oversight over high-impact decisions, traceability of how each decision was made, and an explicit assignment of who answers for it.
These are not exotic controls. They are the version, for AI systems, of a discipline that auditing has known for decades: every decision that produces consequences must be reconstructable and attributable. The new standard does not invent the obligation; it carries it into terrain where we had lost sight of it.
A mature organization is not measured by how much AI it uses, but by how many of its automated decisions it can explain under pressure. The day a regulator, a client, or a court asks who signed, the answer should not be an empty org chart.
Keep reading