There is a comforting story the industry tells about automation. You have a slow, messy, error-prone process. You automate it. Now it is fast, clean, and error-free. The technology does not judge the process; it simply executes it, faster and more consistently than a tired human ever could.

The comforting story is wrong in a specific and dangerous way. Automation is not neutral. Applied to a good process, it is a multiplier of good outcomes. Applied to a bad one, it is a multiplier of bad outcomes, and it can be worse than doing nothing, because it removes the very thing that was quietly keeping the bad process survivable: a human being who noticed.

This is the claim this piece defends. Automating a broken process does not fix it. It industrialises it. It takes a flaw that used to happen occasionally, when someone was rushed or distracted, and makes it happen every single time, instantly, at full scale, with the calm authority of a system that looks like it knows what it is doing. The error rate does not fall. The error rate becomes the design.

The Oldest Warning in the Field Is Still the Right One

None of this is new. The definitive statement of it is thirty-five years old and came from the person who arguably invented the modern discipline of process improvement.

In 1990, Michael Hammer published a piece in the Harvard Business Review with a title that was itself the argument: Reengineering Work: Don't Automate, Obliterate. His central complaint was that companies were spending heavily on information technology and getting disappointing results, and he diagnosed exactly why. They were, he said, using technology to mechanise old ways of doing business. His phrase for it has outlived almost everything else written about the subject: it is time to stop paving the cow paths.

The image is perfect, and worth sitting with. A cow path is the meandering, accidental route that got worn into the ground because some animal wandered that way once and everything followed. Paving it makes it permanent, official, and faster to travel. It does not make it the right route. It embeds a route nobody would ever have designed on purpose into concrete, and then routes all future traffic down it at speed. That is what automating a bad process does. The winding, historically-accidental way your organisation happens to do something gets set in software, and software is a great deal harder to change than habit.

Hammer's point was that the disappointing returns on technology were not a technology failure. They were the entirely predictable result of pointing powerful tools at processes that did not deserve to be preserved. The tool worked. It faithfully preserved and accelerated something that should have been torn up.

Why Automation Removes the Human Safety Net

Hammer explains why automation fails to help. The deeper and more unsettling body of research explains why it can actively hurt, and it comes from the study of how humans and machines share control in places where mistakes kill people: aviation, power plants, chemical process control.

The foundational text is a five-page paper from 1983 by the cognitive psychologist Lisanne Bainbridge, titled Ironies of Automation. It became one of the most cited works in the entire history of human factors engineering, and its argument cuts directly against the comforting story. Automating most of a task, Bainbridge showed, does not remove the human problem. It relocates and often worsens it. The human is left with the leftover scraps the machine could not do, plus a new and punishing job: monitoring the automation for the rare moment it goes wrong.

And humans are consistently poor at that monitoring job, for reasons that are structural rather than a matter of effort or training. The research on what came to be called automation complacency and automation bias, synthesised in a landmark 2010 review by Parasuraman and Manzey, found that when a system runs reliably most of the time, people stop genuinely watching it. They defer to it. They commit errors of omission, failing to notice the system has gone wrong, and errors of commission, actively following the system's incorrect recommendation. Crucially, this was found in experts as much as novices, and it is not reliably eliminated by training or experience, because the mechanism is attentional, not a knowledge gap. Reliable automation lulls the very vigilance it depends on.

Now combine the two findings, because their combination is the whole point. In a manual bad process, the flaw is annoying but visible. A person handles each case, and a person, however imperfect, occasionally looks up and says that does not look right and stops. That human friction, the pausing, the double-take, the informal sanity check, was never in the process documentation, but it was doing enormous unrecorded work. It was the error-correction layer. Automation removes it. The cases now flow through untouched, at speed, and the human who might have caught the problem has been recast as a monitor who, the evidence says, is no longer really watching. You have removed the safety net and hidden the tightrope.

What Happens When You Automate a Flawed Process

Hold the theory against something a property portfolio would actually recognise.

Suppose the process for apportioning a service charge contains a subtle, long-standing error: a category of cost is allocated on the wrong basis, quietly overcharging some tenants and undercharging others. Done manually across a handful of buildings, it is a slow leak. Someone, eventually, queries an invoice. A finance clerk frowns at a number that looks off. The error surfaces, irregularly and late, but it surfaces, because human handling is porous and things leak through the cracks in a way that gets noticed.

Now automate the apportionment across the whole portfolio. The subtle error is no longer occasional. It is applied perfectly, identically, to every building, every unit, every cycle, instantly. The slow leak becomes a portfolio-wide misallocation that is internally consistent and therefore raises no flags. The output looks more correct than the manual version ever did, precisely because it is uniform. It is wrong at scale, and it is wrong with a clean audit trail. The automation did not introduce the error. It took an error the organisation was surviving and made it total.

This is the pattern, not the exception. Automation does not improve a process; it reveals, amplifies, and standardises whatever was already there. If the thing being made precise and consistent is a mistake, you have bought a machine for making that mistake reliably.

The difference becomes clearer when the same flawed workflow is set side by side, before and after automation.

How Automation Changes the Behaviour of a Flawed Process

Why the Clean Output Is the Trap

The most dangerous property of an automated bad process is that it looks better than the manual version it replaced. This is not a side effect. It is the core of the danger.

A messy manual process advertises its own messiness. The inconsistency, the delays, the visible disagreements between people, all of it functions as a signal that the process is imperfect and warrants scrutiny. Automation launders that signal away. It produces uniform, formatted, timely output, and uniformity reads to the human eye as correctness. The very cleanliness that automation is sold on is what disables the scepticism that a bad process needs to survive contact with reality.

So the organisation ends up more confident in a worse outcome. It trusts the number more precisely because a system produced it, at exactly the moment the number deserves less trust, because no human is meaningfully checking it any more. Confidence goes up as correctness goes down. That gap, between how right the output looks and how right it is, is the specific harm, and it is invisible until something forces a reckoning.

What This Means Before You Automate Anything

The takeaway is not that automation is bad. It is a warning about sequence and about honesty regarding what you are actually automating.

Fix the process before you automate it, not after. Automation is the last step in improving a process, never the first. The order matters enormously, because once a process is embedded in software it becomes far harder to change, and its flaws become far harder to see. Automating first is pouring concrete before you have checked the route. Hammer's instruction was to obliterate and redesign before mechanising, and thirty-five years of disappointing technology returns have not made him wrong.

Ask what the humans were silently catching. Before you automate a task, find out what the people doing it manually actually notice and correct that is nowhere in the written procedure. That informal error-correction is usually invisible and usually load-bearing. If you automate the task without replacing that safety net with a deliberate check, you have removed a control you did not know you had.

Distrust the clean output most. When an automated process starts producing tidy, consistent, confident results, treat that as a reason for periodic scrutiny, not a reason to stop looking. Build in the audit that the smoothness will otherwise talk you out of. The output that never looks wrong is the one nobody will ever check, which makes it the most dangerous place for an error to live.

In a property portfolio, the practical version of this is to look, before automating anything, at where staff are already quietly correcting exceptions. The service charge someone always adjusts by hand, the invoice a manager knows to hold back, the renewal that never quite fits the standard flow. Those informal corrections are a map of where the underlying process is weak. Automate over them without redesigning first, and you scale the weakness while deleting the correction. Redesign first, and automation scales the fixed version instead.

Automation is a genuine force multiplier, and that is exactly why it deserves this caution rather than a milder one. A multiplier does not care what it multiplies. Automation never asks whether a process deserves to exist; it simply executes it, faster, more consistently, and with more apparent authority than before. Point it at a good process and it compounds the good. Point it at a bad one and it compounds the bad, just as faithfully and just as fast, with a confident face that makes the damage harder to see. Before you ask whether a process can be automated, the more important question is whether it should be, and the honest answer depends entirely on whether the process was any good to begin with.

Frequently Asked Questions

1. Does automating a bad process make it more efficient?
It makes the process faster and more consistent, but if the process is flawed, that means producing the flawed result faster and more consistently. Efficiency is not the same as improvement. Automation multiplies whatever it is applied to, so applied to a bad process it multiplies the bad outcome rather than fixing it.

2. What did Michael Hammer mean by "don't pave the cow paths"?
In his 1990 Harvard Business Review article, Hammer argued that embedding an outdated, accidental process into software just makes a bad route permanent and faster. His advice was to redesign the process from first principles before applying technology to it, rather than mechanising the existing flawed way of working.

3. What are the "ironies of automation"?
The term comes from Lisanne Bainbridge's 1983 paper. The central irony is that automating most of a task does not remove the human problem; it leaves the human monitoring the system for rare failures, a task humans perform poorly. Automation can therefore create new and more serious problems than the ones it was meant to solve.

4. What is automation complacency?
Automation complacency is the well-documented tendency of people to stop actively scrutinising a system that runs reliably most of the time. Research by Parasuraman and Manzey found it leads to missing failures and to accepting incorrect recommendations, that it affects experts as well as novices, and that it is not reliably eliminated by training, because the cause is attentional rather than a lack of knowledge.

5. Why can an automated process be more dangerous than a manual one?
Because it removes the informal human checking that quietly caught errors, applies any flaw to every case at once, and wraps the result in clean, consistent output that looks authoritative. Confidence in the result rises while actual correctness may fall, and the tidy output discourages the scrutiny that would reveal the problem.

6. Should we never automate a flawed process?
The point is sequence, not prohibition. Fix or redesign the process first, confirm it is sound, identify what human judgment was silently contributing, and only then automate, adding deliberate checks to replace the informal ones you are removing. Automation is powerful and worth using; it is simply the last step, not the first.