Asher's Blog

I was having a conversation with someone who said, “Automation makes mistakes more efficient.”

My first reaction was to disagree. That statement may have described an earlier generation of automation, when a computer simply repeated the rule it had been given. If the rule was wrong, the machine could apply it rapidly and consistently, producing the same mistake thousands of times. But artificial intelligence is becoming something different. Intelligent systems can now compare alternatives, test hypotheses, consult new information, examine research and ask other systems to evaluate their answers. As the technology improves, it should become increasingly capable of noticing an error rather than blindly reproducing it.

So will it still be true that automation makes mistakes more efficient? I think the answer is more complicated than either side of the argument initially appears. AI should eliminate many kinds of error. It can make calculation, monitoring, comparison and detection more reliable than human effort. But intelligence does not automatically remove the possibility of being wrong. It changes where error lives.

The future problem may be less about a machine making a careless arithmetic mistake and more about an intelligent system executing the wrong objective with extraordinary competence. It may have the wrong data, inherit a false assumption, misunderstand a volatile human being or optimise a measurable outcome while damaging something nobody remembered to measure.

Automation may not merely make mistakes more efficient. It may make some mistakes easier to discover and other mistakes more difficult to see.

The difference between repetition and intelligence

Traditional automation is usually deterministic. A rule is written, a condition occurs and the system performs the prescribed action. If every employee in a payroll file receives a five percent increase, the software applies the formula. If the formula is wrong, it can be wrong for everybody at once.

Artificial intelligence introduces a capacity that ordinary automation did not possess. It can work with incomplete information, recognise patterns, generate hypotheses and revise an answer when it receives new evidence. Multiple models can be asked to solve the same problem. One model can criticise another. A system can retrieve a recent paper, run a simulation, compare its result with historical data and flag the uncertainty for human review.

This is a meaningful improvement. In science, finance, manufacturing and medicine, systems that continuously inspect new evidence should catch problems that human beings miss because we are tired, distracted or limited by the amount of information we can process. An intelligent manufacturing system may detect that a tiny temperature change is associated with defective output. A payment platform may recognise a fraud pattern that no employee could identify by reviewing transactions one after another. A payroll system may reconcile deductions across thousands of employees and expose an inconsistency before money is sent.

As models become more capable, many ordinary mistakes should become less common. This is one reason I am optimistic about AI. A system that can check its work, request another opinion and update itself when better evidence arrives is not the same as an old machine mindlessly repeating a command.

But self-correction is not magic. The system must know what it is checking, have access to reliable evidence and possess a valid method for deciding which answer is better. If five models were trained on similar information, rewarded for similar behaviour and asked the same badly framed question, agreement among them may only reveal a shared blind spot.

Five people repeating the same rumour do not transform it into a fact. Five intelligent systems can also be confidently wrong together.

There are at least four kinds of error

It is helpful to separate different mistakes instead of treating error as one thing.

The first is an execution error. The objective is correct, but the task is performed badly. A number is entered incorrectly; a transaction is duplicated. A deadline is missed. AI and automation should become very good at reducing this class of error.

The second is a knowledge error. The system relies on false, incomplete or outdated information. Access to live data, research and independent sources can reduce this risk, but only if the information itself is reliable and the system knows how to weigh conflicting evidence.

The third is an objective error; the system efficiently pursues the wrong goal. A lender optimises for the lowest default rate and excludes every applicant who does not resemble its historically safest borrowers. A hospital optimises for speed and makes patients feel unseen. A company optimises employee output and destroys the trust and creativity on which future performance depends.

The fourth is a human error around the system. A person gives an ambiguous instruction, hides material information, overrides a valid warning, manipulates the data or deploys the system in a situation for which it was never designed.

Greater intelligence may dramatically reduce the first two while leaving the latter two unresolved. In fact, competence at execution can magnify a flawed objective. The machine does not need to misunderstand the instruction. It may understand it perfectly and still create harm because the instruction itself is poor.

This is the issue Stuart Russell explores in Human Compatible: a highly capable machine pursuing an incorrectly specified objective can be more dangerous than a less capable one, precisely because it is better at achieving what it was asked to achieve. The problem is not always insufficient intelligence. Sometimes the problem is that human beings do not know how to express everything they value as an objective.

A well-managed journey in the wrong direction

I recently listened to Travis Kalanick, the founder of Uber, discuss executive hiring. He distinguished between the capacity to organise and manage at scale and the capacity to solve difficult problems. The best executives can do both, but he placed exceptional weight on problem-solving.

I completely agree. A capable manager can organise people, create a plan, assign owners, establish timelines and hold meetings with remarkable discipline. These are valuable abilities. But if the organisation is solving the wrong problem, excellent management can make the journey in the wrong direction look impressively professional.

There will be colourful dashboards. Every task will have an owner. Meetings will start on time. Reports will show progress. The team may deliver precisely what was requested and discover at the end that customers did not want it.

That is what people mean when they warn that automation can make mistakes efficient. Automation behaves like an extraordinarily organised manager. It coordinates the process, removes friction and increases speed. But speed does not prove direction. A faster vehicle going towards the wrong destination creates distance more quickly.

The problem-solver asks a different set of questions: Are we solving the right problem? What evidence would prove that our assumption is false? What has changed since the plan was created? What does the customer actually need? Is the objective producing an unintended consequence? Should we stop?

AI systems are increasingly capable of asking some of these questions. We can instruct one system to attack the assumptions of another. We can use simulations, adversarial tests and independent models. But somebody still has to create an environment in which disagreement is valued. If every AI agent is rewarded for completing the plan rather than questioning it, we will automate management without automating wisdom.

Comparison is useful, but independence matters

The ability of intelligent systems to compare notes is one of the most promising parts of this future. A primary model can produce an answer, a second can investigate weaknesses, a third can verify sources, and a fourth can decide whether the evidence meets a predetermined threshold. In consequential settings, this is far better than accepting the first fluent answer.

However, comparison works best when the reviewers are genuinely independent. If every system uses the same underlying model, the same training data and the same organisational assumptions, their errors may be correlated. They can agree because they learned the same misconception.

This is familiar outside AI. Before a financial crisis, different banks can use similar risk models, respond to the same incentives and reach the same conclusion that an asset is safe. Their agreement does not diversify the risk; it concentrates it. In an African credit market, several lenders may use similar digital footprints and all disadvantage people whose economic lives are poorly represented in formal data. Each model may appear statistically rational, while the entire market reproduces the same exclusion.

The International AI Safety Report examines the risks of increasingly capable general-purpose systems and the difficulty of reliably evaluating and controlling them. The lesson for an ordinary company is not that every AI system will become dangerous. It is that confidence should come from evidence, testing and layered controls—not merely from the fact that a model sounds intelligent.

Good system design needs diversity. Use different sources. Test unusual cases. Invite people from outside the team to challenge assumptions. Compare the model’s prediction with real outcomes after deployment. Make it possible for customers and employees to appeal. Monitor whether small errors are accumulating into a pattern.

An intelligent system should not only compare answers; it should compare itself with reality.

Human beings may remain the least predictable part

The part I am least certain AI can fully resolve is human volatility.

Human beings are not governed by simple laws in the way physical objects are. If an object is released under ordinary conditions near the earth, gravity gives us a highly reliable prediction about what will happen. Human behaviour is different. Psychology, philosophy, economics and sociology give us valuable explanations and probabilities, but they do not produce perfect predictions of an individual decision.

A person may behave consistently for ten years and make a completely different choice on the eleventh. Fear, pride, grief, love, greed, faith, humiliation, ambition or a private conversation can change what somebody does. A customer may say one thing in research and do another when asked to pay. A trusted employee may act out of character under pressure. A political leader may ignore expert advice because the advice threatens their identity or power.

AI will become better at predicting behaviour from patterns. It may know our habits with uncomfortable accuracy. But prediction is not certainty, and the act of predicting people can itself change their behaviour. Once people know how a system judges them, they adapt, perform for it or attempt to manipulate it.

This creates a moving target. In fraud prevention, for instance, the system improves and fraudsters change their methods. The system then learns from the new attacks, and the attackers respond again. There is no permanent formula because intelligent adversaries are part of the environment.

That is why the NIST AI Risk Management Framework treats AI risk as something that must be governed, mapped, measured and managed continuously. A model is not made safe once and left alone. The world around it changes; the people using it change. The incentives change. The system can also be applied to a population or context that was not represented when it was tested.

For a Nigerian company, this is practical rather than theoretical. Customer behaviour can change after a devaluation, a new government policy, a bank failure, a rumour on social media or a sudden loss of trust. A model trained on the last two years may be mathematically sound and still fail to anticipate a new reality. It did not necessarily calculate badly; the environment changed.

The real danger is scale without interruption

The risk in automation is not only that it can make one mistake quickly. It is that it can remove the pauses during which a person might notice that something is wrong.

Manual processes are slow and frustrating, but their friction sometimes creates a review point. Somebody sees an unusual amount, asks a question or recognises a familiar customer’s name. When we automate the process, we may remove the inefficiency and the informal safeguard at the same time.

The answer is not to preserve every manual process. It is to design intentional interruption. A system should know when confidence is low, when an action is irreversible, when the value is unusually high or when the consequence affects a person’s livelihood, freedom or safety. At those points, it should slow down, reveal its reasoning and request appropriate review.

This is particularly important in financial services. If an AI system misclassifies one suspicious payment, there may be a contained loss. If it automatically freezes thousands of legitimate accounts, it can prevent people from buying food, paying salaries or accessing emergency funds. If it sends funds based on an incorrect instruction, speed becomes a liability. The faster the system operates, the stronger reconciliation, limits, permissions and reversal mechanisms must become.

The global Financial Stability Board has repeatedly examined how AI can create operational, cyber and concentration risks in financial services. The concern is not that automation has no value. Financial systems depend on it. The concern is that common providers, shared models and interconnected institutions can allow one failure to travel farther than anybody intended.

Scale should therefore increase humility. The more people a decision can affect, the more aggressively we should search for the ways it could be wrong.

Build problem-solving systems, not only efficient systems

For founders and leaders, the practical lesson is to distinguish between automating a process and solving a problem.

Before automating, define the outcome. Ask why the process exists, who it serves and what must never be sacrificed. Identify the assumptions inside it. Decide what evidence would cause you to change direction. Then automate the parts that benefit from speed, consistency and scale.

After deployment, measure reality rather than activity. Do not celebrate because the system processed one million requests. Ask whether customers received better outcomes, whether errors declined, whether costs fell without transferring harm elsewhere, and whether people can challenge a wrong decision.

Build independent checks. Separate the system that acts from the system that monitors. Reconcile financial actions. Keep logs. Limit permissions. Test extreme and unfamiliar cases. Conduct post-mortems without searching only for somebody to blame. Give human reviewers enough expertise and authority to stop the process.

Most importantly, hire problem-solvers. The future organisation does not need people whose only ability is to move information from one system to another. AI will do that. It needs people who can notice that the system is solving the wrong problem, discover why an outcome changed and redesign the process around new reality.

The best manager of intelligent systems will not merely coordinate their output. That person will know what question deserves to be asked next.

So, does automation make mistakes more efficient?

Sometimes, yes. It can multiply an incorrect rule, scale a biased objective, accelerate a fraudulent instruction or create confidence around a shared false assumption. AI systems comparing notes do not eliminate this possibility, especially when they share data, models, incentives or blind spots.

But that is not the whole future. AI can also make correction more efficient. It can monitor continuously, compare more evidence than any individual can process, expose anomalies early and revise a conclusion when new information appears. Properly designed, it can reduce human inconsistency and prevent mistakes that have always seemed inevitable.

The question is therefore not whether automation is naturally good or naturally dangerous. The question is whether we are building systems that merely execute, or systems that can question, learn, escalate and stop.

I remain optimistic that more intelligent AI will make many mistakes less likely. But I am not convinced that intelligence alone will solve the volatility of human character or the difficulty of choosing the right objective. A machine may understand the entire process and still receive an instruction shaped by fear, greed, pride or incomplete judgment. A human being may also ignore the system when it delivers an answer we do not want to hear.

Perhaps the final challenge will not be teaching machines to avoid mistakes. It will be building people and institutions wise enough not to turn extraordinary intelligence into extraordinarily efficient error.

That future will be interesting to watch. More importantly, it is a future we are already beginning to build.