The Week in Review this week includes the article “Who Do You Trust More: G.I. Joe or A.I. Joe?’. After talking about the Pentagon’s future plans for an AI-driven battlefield, it concludes with a series of doubts about the trustworthiness of computers in an command and control role:

If that comes to pass, doubting some future incarnation of Multivac might be an act of mutiny. Yet there would always be a nagging suspicion: The machine will have been designed by the imperfect species called homo sapiens. What if we got something wrong?

This sort of fearmongering betrays a peculiar view of trust.

Trust is rarely black and white. Outside parent-child, spousal, and, perhaps, human-divine relationships, it’s rare to encounter absolute trust. Even where the level of trust is absolute, more likely than not its domain is constrained. We may absolutely trust our oncologist to cure our cancer, but should treatment fail, we don’t rely on them to tell us what lies in store beyond death.

On the battlefield, the purview of this article, limited degrees and domains of trust are commonplace. The hierarchical nature of the military assumes such a scheme, whereby human commanders are granted differing degrees of authority and autonomy based on their past performance. Even today, while we may think that humans make the decisions about killing or not, invading or not, often the tactical decisions are made by a computer. A pilot may make designate a target and release a weapon, but once released, a bomb may autonomously distinguish friend from foe, as an example. Furthermore, even though the pilot makes the decision about releasing the weapon, the intelligence which identified the target was, most likely, partly artificial.

Just as in programming languages, with time, the arena of human decision will become more abstract. That’s not a bad thing, because it puts uniquely human characteristics to use in the domain where they’re most needed. Perhaps what drives such anxieties as conclude this article is a tendency to resist this flux of responsibility. These worries resemble those that divide labor unions, politicians, and management about outsourcing steel production or software authoring. If some other country or company can roll steel or code cheaper and faster than you, in the long term, you’re better off letting them do it.

This article seems to suggest that we can never make anything more perfect or capable than us. Since we are flawed, the machines we create must be imperfect. Moreover, this argument suggests, their domain of competence will necessarily be a subset of our own. How is it, then, that we commonly judge children more capable than their parents?

The danger lies not so much in whether the machines will make mistakes. Of course they’ll be flawed, as every human commander, whether lieutenant, general, or president, carries his or her own flaws onto the battlefield. If a system comes to be which relies on those elements out of proportion to their flaws, that is a problem, not the existence of the flaws in the first place.

Would I want a computer in control of command and control for a battle theater today? Absolutely not. Would I want it in charge of targeting a weapon? Quite possibly. Nothing prevents a machine from advancing from the second to the first. Nothing, that is, above and beyond the requirements for a human commander to advance: a proven track record in the domain. Silicon or carbon — in a pragmatic philosophy, performance is what matters.

As a final note, neither the original article nor this response address the question of the proper or moral use of the military. For all of history, those decisions have been in the hands of humans, and those humans have made mistakes, sometimes horrific ones On the level that most matters, we seem again and again to have chosen this flawed command and control. Aside from pure pacifists, the mistakes of the past have never led to the suggestion that humans, obviously flawed, be taken out of the loop deciding what are or aren’t moral means and ends of warfare.