Discussion about this post

User's avatar
Mark Ricci's avatar

It's difficult not to be influenced by science fiction in looking at new technologies, especially the revolutionary ones. Two stand out for me: Colossus, the 1966 book by British author DF Jones that was made into a movie called Colossus: The Forbin Project, and Person of Interest, the American TV show created by Jonathan Nolan that ran for five seasons from 2011 to 2016. Both works feature super AIs, and in fact, each has two competing AIs though they resolve their differences in opposite ways. As an aside, I highly recommend watching the Colossus movie and books, and I am a huge fan of Person of Interest which deserved more than its five seasons.

People will always look to worship something because we feel incomplete and not in control, at least at the level most of us find ourselves in life. I imagine it's a lot different being a gazillionaire and not having to worry about where your next paycheck is coming from or can you afford medical care. But when you are not in control of most things, you make concessions to who is in control of the things you want in the hope that will get or continue to get those things. No surprise that ancient religions began with worship of the natural elements hoping to get more sunshine, more rain when needed, more light at night.

Fast forward to countries where the state controls everything and you have worship of the leader or the state. Typically in such states, it's not simply that things will not flow to you if you act otherwise, but you may not live long enough to suffer deprivation. So there naturally exists a prudent stay on the good side of the state, but someone either spontaneously or by direction, proposes an enhanced version of staying on the good side of the state by worshipping it. On second thought, the spontaneous worship comes later, if at all, but the direction by the state is instituted pretty quickly after it takes control in the form of loyalty oaths because those leading the state realize they need people to carry our their plans and it metastasizes from there.

Just think back five years ago to 2020 and how full-blown Covid turned the world into a sickening oblate spheroid of control and collaboration with control by the various nations and their health authorities with hordes of people willing to spy and report on their neighbors for not wearing masks. These weren't people concerned about their health, but concerned about serving who or what was in power. Now imagine an AI being the one in power, controlling multiple elements of people's lives including doling out or denying benefits. Too many people will sign up to do its bidding and more than a few people will join the inevitable new religion of the AI.

Actually, there precedent for such an AI religion in China's Social Credit System that monitors behaviors to encourage people to consciously practice the socialist core values of the CCP and punishes those whose scores are low meaning they do not comply with the CCP's wishes. It's a massive amount of surveillance and already done using computers, but throw AI into it not just to assist, not to just monitor, but to hand down the criteria for high scores and it will inevitably conclude that requiring people to worship it or they get nothing and have nothing will be the way to go.

I hope I'm wrong.

Expand full comment
Alejandro Piad Morffis's avatar

Fascinating read. It also echoes more than one sci-fi story, and sci-fi is often prescient. While I'm reasonably skeptic on the prospect of superintelligence arriving anytime soon, I definitely agree that, if we were to have it tomorrow, this is the way most people would react.

Expand full comment
2 more comments...

No posts