The Madness of Tunnel Vision Utilitarianism
Mary Harrington has published a Substack piece entitled “The Sorcerer's Apprentice,” in which she reckons with the unyielding march of technological innovation. It reminded me of a section of our conversation that I’d left on the cutting room floor because it felt too convoluted to include in a discussion that already stretched the limits of coherence. Her piece has inspired me to revive the section by combining it with a Disney animation and propping it up with some accompanying text.
For newcomers to The Process, I sat down with Mary Harrington to discuss her concept of the Cyborg Theocracy, a fascinating frame to describe the amorphous technocratic regime that’s evolving through the institutions of the Western world.
Before the revived video below, Mary and I had been talking about contemporary social movements and their moral structures. We touched on Effective Altruism, a movement committed to using evidence and reason to 'optimise’ the world for well-being. EA is disembodied by design and tends toward utilitarian approaches to moral conduct.
It’s popular among New Atheists, tech entrepreneurs, and Silicon Valley autists, perhaps due to its rationalist approach and laser focus on quantifiable metrics.
An interesting case study for the problems embedded within EA comes from one of its most prominent figures, Sam Bankman-Fried. He stands accused of one of the biggest financial frauds in U.S. history after building a successful cryptocurrency exchange and allegedly stealing billions of dollars of customer deposits to make charitable donations and fund high-risk investments.
Sam's endlessly professed commitment to EA has led many, including myself, to speculate that he’d been morally offsetting his alleged criminal behaviour against philanthropic endeavours. It’s fair to call this presumptive because fraud can be found in all moral and religious circles. However, if you listen to interviews with Sam prior to his arrest, it’s clear that he thinks deeply, and perhaps even obsessively, about these matters. A rationale of 'the ends justify the means' aligns with the disembodied EA framework, which neglects the fuzzier aspects of ethics like duty and virtue.
Even the brains trust behind the EA movement, a group of philosophers who spend their days contemplating moral trade-offs, have been accused of nonfeasance. They apparently turned a blind eye to Sam’s dodgy business dealings while stewarding more than 160 million dollars from his Future Fund into their EA causes.
At the beginning of the video at the bottom of this piece, I propose that a similar species of machine-like calculus was applied to the complex moral problem of COVID-19 in my home city of Melbourne, Victoria. There I saw state power fully mobilised to the dogged pursuit of a simplistic metric and a bizarre deferral of all ground-level sensemaking to a dissociated central authority.
While science can inform decision-making, for example by quantifying the risks involved in different courses of action and evaluating the trade-offs between them, it doesn’t actually lead anywhere. Once you decide to do something with the facts offered up by the scientific method you’ve stepped out of the scientific domain. When a leader claims their political decisions are scientific, they’re either confused about the scientific method or preemptively skirting responsibility for the outcome of their choices.
The Victorian leadership did just this by leaning into the authority of experts who took on the role of being the representatives for science. These public health officials presented us with a dizzying array of statistical arrangements and rationalisations for their actions, which frequently contradicted facts uncovered by the real scientific method that was going on quietly behind the scenes.
Their opinions took on the authority of holy writ and our attempts to make sense of the trade-offs were suppressed by partisan zealots, sophisticated ‘disinformation' suppression techniques, and in some cases, the police.
As weeks turned into months, the people around me became confused and listless, and even those who’d been enthusiastically supportive of the measures began to grapple with internal dissonance. For all their apparent quantitative success, the Victorian leadership had created a qualitative hellscape.
In the end, we were held for 262 days under the most extreme lockdown measures in the Western world.
An over-the-top faith in science can transform what is essentially a methodological approach into the ideology of Scientism. Reason too, when not adequately balanced against humility, can turn into a form of madness. In both cases, human existence is oversimplified and aspects of reality that can’t be easily measured or quantified are dismissed.
Similar to the accusations levied against Bankman-Fried, the Victorian government was engaged in a form of tunnel vision utilitarianism. They defined zero community transmission as their metric, which was effectively a simplistic stand-in for the impossibly complex notion of ‘well-being’, and then mechanically pursued their goal.
In the clip below, I call this ‘paperclip morality’, after the paperclip maximiser thought experiment.
The point here is not to attack science and reason, but to illuminate how they are neither scientific nor reasoned when pushed beyond their limits.
Perhaps the Melbourne lockdowns and the tragic tale of Bankman-Fried can join The Sorcerer’s Apprentice as allegories for what Mary identifies in her piece as the core struggle of modernity: Given our limitations as human beings, how are we to come to terms with the power of science?
She steps with trepidation into a potential answer and I strongly recommend the read.