ArticleIs the AI carnival over? Our AI barons are publicly concerned about whether the risk to ongoing human existence is too great.
The early railroad barons of the Nineteenth Century showed us the dangers of being reckless with new technology, and the cost to human life. The first American railroads had a terrible reputation for disasters and death tolls, with many European tourists terrified to board an American train.
Rachel Felix, a French actress undertaking a railroad tour of the United States in 1855–1856, found it “quite inconceivable how few precautions are taken to avert accidents.” She thought it was “folly” to have only one track and she said the “the grading is not solid, the bridges . . . thrown together in haste [and] built of bad timbers.”
It was more than bad haste in construction on the American railroads – historian Mark Aldrich observed in Death Rode the Rails (2006) that there was sometimes a deliberate trade-off between capital and human safety. Gates and gatekeepers were not installed, and cheap natural resources such as flimsy wooden trestles were substituted for expensive investment wherever possible, with disastrous consequences.
When masses of soldiers returned home from the American Civil War, people started wondering whether living with this risk to life and safety should be accepted as the wallpaper to their everyday life. They decided that regulation was needed, and the Railroad Safety Appliance Act was enacted in 1893, leading to a sharp drop in railroad accidents in the 20th century.
Tech giant Anthropic recently revised its AI risk assessment from “very low” to “low but not negligible”. The revised assessment might not raise any eyebrows at first glance – we are still in low-risk territory. That is until we realise that the risk is for “catastrophic harm”, leading to fundamental destabilisation of global systems or existential threat.
The CEOs from Anthropic and OpenAI are now suggesting a collective “go slow” on the development of new technology, publicly concerned about the risk. These two rival companies dominate the contemporary AI landscape, and an initial public offering is expected for both in the near future. Should we be uneasy at their public agreement on this point, and their desire to slow things down in tandem?
I was a competition lawyer in the early 2000s, and at one competition law conference our team of lawyers all wore t-shirts with the following words from the economist Adam Smith (1723-1790) printed on the back: “People of the same trade seldom meet together, even for merriment and diversion, but [when they do] the conversation ends in a conspiracy against the public, or in some contrivance to raise prices.”
As the junior lawyers in my team trained each other up as experts in competition law, I remember many sessions discussing the “conspiracy” between the early American railroad barons. As the barons built their railroads with “bad timbers” they were also busy fixing freight rates with their competitors, pooling traffic and charging farmers high prices. Public outcry led to the Sherman Act 1890, the first federal antitrust regulation in the United States.
We’ve continued on to create a venture capital system that rewards techno-disruptors and that has facilitated the rapid growth of AI companies. Adam Smith would say that it makes sense for the entrepreneurs at the head of this technological pack to collude in order to raise the barrier for those entrepreneurs lagging a bit behind, to reduce output as a “contrivance to raise prices.”
It’s also a good time to remember Smith’s warning that our business leaders will not have our public wellbeing and safety front of mind when they are in the same room (or zoom) together. They are more likely to be figuring out how they can accumulate more wealth, take life easy, and protect themselves from the threat of competition and/or regulation.
In my last week as a competition lawyer I attended the firm’s Christmas party. It was carnival-themed, with roaming fire-breathers and a big tent pitched in Sydney’s botanical gardens. In fact, the carnival was nearly over: the Global Financial Crisis would soon snuff out this type of lavish corporate display.
As we abandon our “old” technologies and make room for the new, it’s a good time to remember that we should always be wary of constructing a new world for ourselves with bad timbers. What measures do we need to put into place to protect life and allow our communities to flourish?
Jesus raised the red flag on our perpetual human drive to accumulate, noting that “For what will it profit a man if he gains the whole world, but forfeits his soul?” The drive to make profit from AI innovation may well be putting our collective souls in danger.
Many fear the AI techno-carnival may nearly be over. But perhaps we can still decide which carnival that will be. The carnival of endless innovation, guardrails ignored, with AI breathing its technological fire into our economy and accumulating ever more wealth for our barons? Or the carnival of life as we know it, with AI wreaking catastrophic harm?