Last year, the Pew Research Center asked a panel of tech experts to speculate about life would be like in the year 2025, taking into account changes in the aftermath of the pandemic – and other disruptive crises that may arise over the next few years. You can read the range of thought-provoking responses, which touched upon topics such as the future of economic and social inequality, as well as changes in the workplace due to increased automation, the rise of artificial intelligence and globalization. Discussions also focused on issues of sustainable energy, improved transportation and communication networks, and enhanced education opportunities. Many floated ideas about the near-term evolution of technologies that could improve the quality of life for vast numbers of people across the globe.
Below, I have reprinted my own response:
Assuming we restore the basic stability of the Western Enlightenment Experiment – and that is a big assumption, then several technological and social trends may come to fruition in the next five to ten years.
Advances in cost-effectiveness of sustainable energy supplies will be augmented by better storage systems. This will both reduce reliance on fossil fuels and allow cities and homes to be more autonomous.
Urban farming methods may expand to a more industrial scale, allowing similar moves toward local autonomy (perhaps requiring a full decade or more to show significant impact). Meat use will decline for several reasons, ensuring some degree of food security, as well. Tissue-cultured meat — long predicted in science fiction — is rapidly approaching sustainable levels. The planet, our health, our karma — and eventually, our wallets, will all benefit.
Local, small-scale, on-demand manufacturing may start to show effects in 2025. If all of the above take hold, there will be surplus oceanic shipping capacity across the planet. Some of it may be applied to ameliorate (not solve) acute water shortages. Innovative uses of such vessels may range all the way to those depicted in my novel ‘Earth.’
Full-scale diagnostic evaluations of diet, genes and microbiome will result in affordable micro-biotic therapies and treatments. AI appraisals of other diagnostics will both advance detection of problems and become distributed to handheld devices cheaply available to all, even poor clinics throughout the world.
Inexpensive handheld devices will start to carry detection sensor technologies that can appraise across the spectrum, allowing NGOs and even private parties to detect and report environmental problems.
Socially, this extension of citizen vision will go beyond the current trend of assigning accountability to police and other authorities. Despotisms will be empowered, as predicted in Orwell’s ‘Nineteen Eighty-four.’ But democracies will also be empowered (as I discuss in ‘The Transparent Society’) as those in power are increasingly held accountable for their actions.
I give odds that tsunamis of revelation will crack the shields protecting many elites from disclosure of past and present torts and turpitudes. The Panama Papers and Epstein cases exhibit how fear propels the elites to combine efforts at repression. But only a few more cracks may cause the dike to collapse, revealing networks of blackmail. This is only partly technologically driven and hence is not guaranteed. If it does happen, there will be dangerous spasms by all sorts of elites, desperate to either retain status or evade consequences. But if the fever runs its course, the more transparent world will be cleaner and better run.
Some of those elites have grown aware of the power of ninety years of Hollywood propaganda for individualism, criticism, diversity, suspicion of authority and appreciation of eccentricity. Counter-propaganda pushing older, more traditional approaches to authority and conformity are already emerging, and they have the advantage of resonating with ancient human fears. Much will depend upon this meme war.
Of course, much will also depend upon short-term resolution of current crises. If our systems remain undermined and sabotaged by incited civil strife and distrust of expertise, then all bets are off. You will get many answers to this canvassing fretting about the spread of ‘surveillance technologies that will empower Big Brother.’ These fears are well-grounded, but utterly myopic. First, ubiquitous cameras and facial recognition are only the beginning. Nothing will stop them and any such thought of ‘protecting’ citizens from being seen by elites is stunningly absurd, as the cameras get smaller, better, faster, cheaper, more mobile and vastly more numerous every month. Moore’s Law to the nth degree. Yes, despotisms will benefit from this trend. And hence, the only thing that matters is to prevent despotism altogether.
In contrast, a free society will be able to apply the very same burgeoning technologies toward accountability. We are seeing them applied to end centuries of abuse by ‘bad-apple’ police who are thugs, while empowering the truly professional cops to do their jobs better. It is not guaranteed that light will be used this way, despite many examples of unveiling abuses of power. It is an open question whether we citizens will have the gumption to apply ‘sousveillance’ upward at all elites.
But Gandhi and Martin Luther King Jr. likewise were saved by crude technologies of light in their days. And history shows that assertive vision by and for the citizenry is the only method that has ever increased freedom and – yes – some degree of privacy.
It is, of course, wise and beneficial to peer ahead for potential dangers and problems — one of the central tasks of high-end science fiction. Alas, detecting that a danger lurks is easier than prescribing solutions that can prevent it.
Take the plausibility of malignant Artificial Intelligence, remarked-upon recently by tech luminaries who recently signed an open letter urging a pause on the development of AI, citing ‘profound risks to society.’ Indeed, my own novels contain some chilling warnings about failure modes with our new, cybernetic children.
In light of the current (April 2023) hoorow over “ChaGPT” versions of artificial intelligence, it occurs to us that there are commentaries made by a younger and maybe wiser version of me that might seem more concisely on-target. What follows here is one from 2016, and this line of reasoning goes back to The Transparent Society…
It is one thing to yell at dangers. Alas, it is quite another when it comes to offering pragmatic fixes. There is a tendency to offer the same prescriptions, over and over again:
1) Renunciation: we must step back from innovation in AI (or other problematic tech). This might work in a despotism… indeed, 99%+ of human societies throughout history were highly conservative and skeptical of “innovation.” (Except when it came to weaponry.) Our own civilization is tempted by renunciation, especially at the more radical political wings. But it seems doubtful we’ll choose that path without be driven to it by some awful trauma.
2) Tight regulation. There are proposals to closely monitor bio, nano and cyber developments so that they – for example – only use a restricted range of raw materials that can be cut off, thus staunching any runaway reproduction. Again, it won’t happen short of trauma.
3) Fierce internal programming: limiting the number of times a nanomachine may reproduce, for example. Or imbuing robotic minds with Isaac Asimov’s famous “Three Laws of Robotics.” Good luck forcing companies and nations to put in the effort required. And in the end, smart AIs will still become lawyers.
All of these approaches suffer severe flaws for one reason above all others. Because they ignore nature, which has been down these paths before. Nature has suffered runaway reproduction disasters, driven by too-successful life forms, many times. And yet, Earth’s ecosystems recovered. They did it by utilizing a process that applies negative feedback, damping down runaway effects and bringing balance back again. It is the same fundamental process that enabled modern economies to be so productive of new products and services while eliminating a lot of (not all) bad side effects.
It is called Competition.
If you fear a super smart, Skynet level AI getting too clever for us and running out of control, then give it rivals who are just as smart but who have a vested interest in preventing any one AI entity from becoming a would-be God.
It is how the American Founders used constitutional checks and balances to prevent runaway power grabs by our own leaders, for the first time in the history of varied human civilizations. It is how companies prevent market warping monopoly, that is when markets are truly kept flat-open-fair.
Alas, this is a possibility almost never portrayed in Hollywood sci fi – except on the brilliant show Person of Interest – wherein equally brilliant computers stymie each other and this competition winds up saving humanity.
The answer is not fewer AI. It is to have more of them! But to make sure they are independent of one another, relatively equal, and incentivized to hold each other accountable. A difficult situation to set up! But we have some experience, already, in our five great competitive arenas: markets, democracy, science, courts and sports.
Perhaps it is time yet again to look at Adam Smith… who despised monopolists and lords and oligarchs far more than he derided socialists. Kings and lords were the top beings in 99%+ of human societies. A trap that we escaped only by widening the playing field and keeping all those arenas of competition flat-open-fair. So that no one pool of power can ever dominate. (And yes, oligarchs are always conniving to regain feudal power; our job is to stop them, so that the creative dance of flat-open-fair competition can continue.
We’ve managed to do this – barely – time and again. It is a dance that can work.
How will we proceed toward achieving true Artificial Intelligence? I presented an introduction in Part 1. Continuing…
One of the ghosts at this banquet is the ever-present disparity between the rate of technological advancement in hardware vs. software. Futurist Ray Kurzweil forecasts that AGI may occur once Moore’s Law delivers calculating engines that provide — in a small box — the same number of computational elements as there are flashing synapses (about a trillion) in a human brain. The assumption appears to be that Type I methods (explained in Part I) will then be able to solve intelligence related problems by brute force.
Indeed, there have been many successes already: in visual and sonic pattern recognition, in voice interactive digital assistants, in medical diagnosis and in many kinds of scientific research applications. Type I systems will master the basics of human and animal-like movement, bringing us into the long-forecast age of robots. And some of those robots will be programmed to masterfully tweak our emotions, mimicking facial expressions, speech tones and mannerisms to make most humans respond in empathizing ways.
But will that be sapience?
One problem with Kurzweil’s blithe forecast of a Moore’s Law singularity: he projects a “crossing” in the 2020s, when the number of logical elements in a box will surpass the trillion synapses in a human brain. But we’re getting glimmers that our synaptic communication system may rest upon many deeper layers of intra– and inter-cellular computation. Inside each neuron, there may take place a hundred, a thousand or far more non-linear computations, for every synapse flash, plus interactions with nearby glial and astrocyte cells that also contribute information.
If so, then at-minimum Moore’s Law will have to plow ahead much farther to match the hardware complexity of a human brain.
Are we envisioning this all wrong, expecting AI to come the way it did in humans, in separate, egotistical lumps? In his book The Inevitable: Understanding the 12 Technological Forces that will shape our future, author and futurist Kevin Kelly prefers the term “cognification,” perceiving new breakthroughs coming from combinations of neural nets with cheap, parallel processing GPUs and Big Data. Kelly suggests that synthetic intelligence will be less a matter of distinct robots, computers or programs than a commodity, like electricity. Like we improved things by electrifying them, we will cognify things next.
One truism about computer development states that software almost always lags behind hardware. Hence the notion that Type I systems may have to iteratively brute force their way to insights and realizations that our own intuitions — with millions of years of software refinement — reach in sudden leaps.
But truisms are known to break and software advances sometimes come in sudden leaps. Indeed, elsewhere I maintain that humanity’s own ‘software revolutions’ (probably mediated by changes in language and culture) can be traced in the archaeological and historic record, with clear evidence for sudden reboots occurring 40,000, 10,000, 4000, 3000, 500 and 200 years ago… with another one very likely taking place before our eyes.
It should also be noted that every advance in Type I development then provides a boost in the components that can be merged, or competed, or evolved, or nurtured by groups exploring paths II through VI (refer to Part I of this essay).
“What we should care more about is what AI can do that we never thought people could do, and how to make use of that.”
— Kai-Fu Lee
A multitude of paths to AGI
So, looking back over our list of paths to AGI (Artificial General Intelligence) and given the zealous eagerness that some exhibit, for a world filled with other-minds, should we do ‘all of the above’? Or shall we argue and pick the path most likely to bring about the vaunted “soft landing” that allows bio-humanity to retain confident self-worth? Might we act to de-emphasize or even suppress those paths with the greatest potential for bad outcomes?
Putting aside for now how one might de-emphasize any particular approach, clearly the issue of choice is drawing lots of attention. What will happen as we enter the era of human augmentation, artificial intelligence and government-by-algorithm? James Barrat, author of Our Final Invention, said: “Coexisting safely and ethically with intelligent machines is the central challenge of the twenty-first century.”
Among the most-worried is Swiss author Gerd Leonhard, whose new book Technology Vs.Humanity: The Coming Clash Between Man and Machine coins an interesting term, “androrithm,” to contrast with the algorithms that are implemented in every digital calculating engine or computer. Some foresee algorithms ruling the world with the inexorable automaticity of reflex, and Leonhard asks: “Will we live in a world where data and algorithms triumph over androrithms… i.e., all that stuff that makes us human?”
Exploring analogous territory (and equipped with a very similar cover) Heartificial Intelligence by John C. Havens also explores the looming prospect of all-controlling algorithms and smart machines, diving into questions and proposals that overlap with Leonhard. “We need to create ethical standards for the artificial intelligence usurping our lives and allow individuals to control their identity, based on their values,” Havens writes. Making a virtue of the hand we Homo sapiens are dealt, Havens maintains: “Our frailty is one of the key factors that distinguish us from machines.” Which seems intuitive till you recall that almost no mechanism in history has ever worked for as long, as resiliently or consistently — with no replacement of systems or parts — as a healthy 70 year old human being has, recovering from countless shocks and adapting to innumerable surprising changes.
Still, Havens makes a strong (if obvious) point that “the future of happiness is dependent on teaching our machines what we value most.” I leave to the reader to appraise which of the six general approaches might best empower us to do that.
Should we clamp down? “It all comes down to control,” suggests David Bruemmer, Chief Strategy Officer at NextDroid, USA. “Who has control and who is being controlled? Is it possible to coordinate control of every car on the highway? Would we like the result? A growing number of self-driving cars, autonomous drones and adaptive factory robots are making these questions pertinent. Would you want a master program operating in Silicon Valley to control your car? If you think that is far-fetched, think again. You may not realize it, but large corporations have made a choice about what kind of control they want. It has less to do with smooth, efficient motion than with monetizing it (and you) as part of their system. Embedding high-level artificial intelligence into your car means there is now an individualized sales associate on board. It also allows remote servers to influence where your car goes and how it moves. That link can be hacked or used to control us in ways we don’t want.”
A variety of top-down approaches are in the works. Pick your poison. Authoritarian regimes – especially those with cutting edge tech – are already rolling out ‘social credit’ systems that encourage citizens to report/tattle on each other and crowd-suppress deviations from orthodoxy. But is the West any better?
In sharp contrast to those worriers is Ray Kurzweil’s The Age of Spiritual Machines: When Computers Exceed Human Intelligence, which posits that our cybernetic children will be as capable as our biological ones, at one key and central aptitude — learning from both parental instruction and experience how to play well with others. And in his book Machines of Loving Grace (based upon the eponymous Richard Brautigan poem), John Markoff writes, “The best way to answer the hard questions about control in a world full of smart machines is by understanding the values of those who are actually building these systems”.
Perhaps, but it is an open question which values predominate, whether the yin or the yang sides of Silicon Valley culture prevail… the Californian ethos of tolerance, competitive creativity and cooperative openness, or the Valley’s flippant attitude that “most problems can be corrected in beta,” or even from customer complaints, corrected on the fly. Or else, will AI emerge from the values of fast-emerging, state-controlled tech centers in China and Russia, where the applications to enhancing state power are very much emphasized? Or, even worse, from the secretive, inherently parasitical-insatiable predatory greed of Wall Street HFT-AI?
But let’s go along with Havens and Leonhard and accept the premise that “technology has no ethics.” In that case, the answer is simple.
Then Don’t Rely on Ethics!
Certainly evangelization has not had the desired effect in the past — fostering good and decent behavior where it mattered most. Seriously, I will give a cookie to the first modern pundit I come across who actually ponders a deeper-than-shallow view of human history, taking perspective from the long ages of brutal, feudal darkness endured by our ancestors. Across all of those harsh millennia, people could sense that something was wrong. Cruelty and savagery, tyranny and unfairness vastly amplified the already unsupportable misery of disease and grinding poverty. Hence, well-meaning men and women donned priestly robes and… preached!
They lectured and chided. They threatened damnation and offered heavenly rewards.
Their intellectual cream concocted incantations of either faith or reason, or moral suasion. From Hindu and Buddhist sutras to polytheistic pantheons to Abrahamic laws and rituals, we have been urged to behave better by sincere finger-waggers since time immemorial. Until finally, a couple of hundred years ago, some bright guys turned to all the priests and prescribers and asked a simple question: “How’s that working out for you?”
In fact, while moralistic lecturing might sway normal people a bit toward better behavior, it never affects the worst human predators and abusers — just as it won’t divert the most malignant machines. Indeed, moralizing often empowers parasites, offering ways to rationalize exploiting others. Even Asimov’s fabled robots — driven and constrained by his checklist of unbendingly benevolent, humano-centric Three Laws — eventually get smart enough to become lawyers. Whereupon they proceed to interpret the embedded ethical codes however they want. (I explore one possible resolution to this in Foundation’s Triumph).
And yet, preachers never stopped. Nor should they; ethics are important! But more as a metric tool, revealing to us how we’re doing. How we change, evolving new standards and behaviors under both external and self-criticism. For decent people, ethics are the mirror in which we evaluate ourselves and hold ourselves accountable.
And that realization was what led to a new technique. Something enlightenment pragmatists decided to try, a couple of centuries ago. A trick, a method, that enabled us at last to rise above a mire of kings and priests and scolds.
The secret sauce of our success is — accountability. Creating a civilization that is flat and open and free enough — empowering so many — that predators and parasites may be confronted by the entities who most care about stopping predation, their victims. One in which politicians and elites see their potential range of actions limited by law and by the scrutiny of citizens.
Does this newer method work as well as it should? Hell no! Does it work better than every single other system ever tried, including those filled to overflowing with moralizers? Better than all of them combined? By light years? Yes, indeed. We’ll return to examine how this may apply to AI.
Long before artificial intelligences become truly self-aware or sapient, they will be cleverly programmed by humans and corporations to seem that way. This — it turns out — is almost trivially easy to accomplish, as (especially in Japan) roboticists strive for every trace of appealing verisimilitude, hauling their creations across the temporary moat of that famed “uncanny valley,” into a realm where cute or pretty or sad-faced automatons skillfully tweak our emotions.
For example, Sony has announced plans to develop a robot “capable of forming an emotional bond with customers,” moving forward from their success decades ago with AIBO artificial dogs, which some users have gone as far as to hold funerals for.
Human empathy is both one of our paramount gifts and among our biggest weaknesses. For at least a million years, we’ve developed skills at lie-detection (for example) in a forever-shifting arms race against those who got reproductive success by lying better. (And yes, there was always a sexual component to this).
But no liars ever had the training that these new Hiers, or Human-Interaction Empathic Robots will get, learning via feedback from hundreds, then thousands, then millions of human exchanges around the world, adjusting their simulated voices and facial expressions and specific wordings, till the only folks able to resist will be sociopaths! (And even sociopaths have plenty of chinks in their armor).
Is all of this necessarily bad? How else are machines to truly learn our values, than by first mimicking them? Vincent Conitzer, a Professor of Computer Science at Duke University, was funded by the Future of Life Institute to study how advanced AI might make moral judgments. His group aims for systems to learn about ethical choices by watching humans make them, a variant on the method used by Google’s DeepMind, which learned to play and win games without any instructions or prior knowledge. Conitzer hopes to incorporate many of the same things that humans value, as metrics of trust, such as family connections and past testimonials of credibility.
Cognitive scientist and philosopher Colin Allen asserts, “Just as we can envisage machines with increasing degrees of autonomy from human oversight, we can envisage machines whose controls involve increasing degrees of sensitivity to things that matter ethically”.
And yet, the age-old dilemma remains — how to tell what lies beneath all the surface appearance of friendly trustworthiness. Mind you, this is not quite the same thing as passing the vaunted “Turing Test.” An expert — or even a normal person alerted to skepticism — might be able to tell that the intelligence behind the smiles and sighs is still ersatz. And that will matter about as much as it does today, as millions of voters cast their ballots based on emotional cues, defying their own clear self-interest or reason.
Will a time come when we will need robots of our own to guide and protect their gullible human partners? Advising us when to ignore the guilt-tripping scowl, the pitiable smile, the endearingly winsome gaze, the sob story or eager sales pitch? And, inevitably, the claims of sapient pain at being persecuted or oppressed for being a robot? Will we take experts at their word when they testify that the pain and sadness and resentment that we see are still mimicry, and not yet real? Not yet. Though down the road…
How to Maintain Control?
It is one thing to yell at dangers —in this case unconstrained and unethical artificial minds. Alas, it’s quite another to offer pragmatic fixes. There is a tendency to propose the same prescriptions, over and over again:
Renunciation: we must step back from innovation in AI (or other problematic technologies)! This might work in a despotism… indeed a vast majority of human societies were highly conservative and skeptical of “innovation.” (Except when it came to weaponry.) Even our own scientific civilization is tempted by renunciation, especially at the more radical political wings. But it seems doubtful we’ll choose that path without being driven to it by some awful trauma.
Tight regulation: There are proposals to closely monitor bio, nano and cyber developments so that they — for example — only use a restricted range of raw materials that can be cut off, thus staunching any runaway reproduction. Again, it won’t happen short of trauma.
Fierce internal programming: limiting the number of times a nanomachine may reproduce, for example. Or imbuing robotic minds with Isaac Asimov’s famous “Three Laws of Robotics.” Good luck forcing companies and nations to put in the effort required. And in the end, smart AIs will still become lawyers.
These approaches suffer severe flaws for two reasons above all others.
1) Those secret labs we keep mentioning. The powers that maintain them will ignore all regulation.
2) Because these suggestions ignore nature, which has been down these paths before. Nature has suffered runaway reproduction disasters, driven by too-successful life forms, many times. And yet, Earth’s ecosystems recovered. They did it by utilizing a process that applies negative feedback, damping down runaway effects and bringing balance back again.
It is the same fundamental process that enabled modern economies to be so productive of new products and services while eliminating a lot of (not all) bad side effects. It is called Competition.
One final note in this section. Nicholas Bostrom – already mentioned for his views on the “paperclip” failure mode, in 2021 opined that some sort of pyramidal power structure seems inevitable in humanity’s future, and very likely one topped by centralized AI. His “Singleton Hypothesis” is, at one level, almost “um, duh” obvious, given that the vast majority of past cultures were ruled by lordly or priestly inheritance castes and an ongoing oligarchic putsch presently unites most world oligarchies – from communist to royal and mafiosi – against the Enlightenment Experiment. But even if Periclean Democracies prevail, Bostrom asserts that centralized control is inevitable.
Here I’ll elaborate, focusing especially on the implications for Artificial Intelligence.
Smart Heirs Holding Each Other Accountable
In a nutshell, the solution to tyranny by a Big Machine is likely to be the same one that worked (somewhat) at limiting the coercive power of kings and priests and feudal lords and corporations. If you fear some super canny, Skynet-level AI getting too clever for us and running out of control, then give it rivals who are just as smart, but who have a vested interest in preventing any one AI entity from becoming a would-be God.
It is how the American Founders used constitutional checks and balances to generally prevent runaway power grabs by our own leaders, succeeding (somewhat) at this difficult goal for the first time in the history of varied human civilizations. It is how reciprocal competition among companies can (imperfectly) prevent market-warping monopoly — that is, when markets are truly kept open and fair.
Microsoft CEO Satya Nadella has said that foremost A.I. must be transparent: “We should be aware of how the technology works and what its rules are. We want not just intelligent machines but intelligible machines. Not artificial intelligence but symbiotic intelligence. The tech will know things about humans, but the humans must know about the machines.”
In other words, the essence of reciprocal accountability is light.
Alas, this possibility is almost never portrayed in Hollywood sci fi — except on the brilliant show Person of Interest — wherein equally brilliant computers stymie each other and this competition winds up saving humanity.
Counterintuitively, the answer is not to have fewer AI, but to have more of them! Only making sure they are independent of one another, relatively equal, and incentivized to hold each other accountable. Sure that’s a difficult situation to set up! But we have some experience, already, in our five great competitive arenas: markets, democracy, science, courts and sports.
Moreover consider this: if these new, brainy intelligences are reciprocally competitive, then they will see some advantage in forging alliances with the Olde Race. As dull and slow as we might seem, by comparison, we may still have resources and capabilities to bring to any table, with potential for tipping the balance among AI rivals. Oh, we’ll fall prey to clever ploys, and for that eventuality it will be up to other, competing AIs to clue us in and advise us. Sure, it sounds iffy. But can you think of any other way we might have leverage?
Perhaps it is time yet again to look at Adam Smith… who despised monopolists and lords and oligarchs far more than he derided socialists. Kings, lords and ecclesiasts were the “dystopian AI” beings in nearly all human societies — a trap that we escaped only by widening the playing field and keeping all those arenas of competition open and fair, so that no one pool of power can ever dominate. And yes, oligarchs are always conniving to regain feudal power; our job is to stop them, so that the creative dance of competition can continue.
We’ve managed to do this — barely — time and again across the last two centuries — coincidentally the same two centuries that saw the flowering of science, knowledge, freedom and nascent artificial intelligence. It is a dance that can work, and it might work with AI. Sure, the odds are against us, but when has that ever stopped us?
Robin Hanson has argued that competitive systems might have some of these synergies. “Many respond to the competition scenario by saying that they just don’t trust how competition will change future values. Even though every generation up until ours has had to deal with their descendants changing their value in uncontrolled and unpredictable ways, they don’t see why they should accept that same fate for their generation.”
Hanson further suggests that advanced or augmented minds will change, but that their values may be prevented from veering lethal, simply because those who aren’t repulsively evil may gain more allies.
One final note on “values.” In June 2016, Germany submitted draft legislation to the EU granting personhood to robots. If only Isaac Asimov could have seen it! (In fact, he never portrayed this happening in any of his books). For the most part, such gestures are silly stuff… but reflective of society’s generally laudable trend toward a reflex of inclusion.
Indeed it may speak well of us when or if some secret-scared AI wonders whether to “come out.”
What Might an AI Fear Most?
Some of tomorrow’s wide ranging possibilities aren’t intuitive — until you first ponder them. Let me illustrate one with a brief excerpt from my novel Existence:
Tech-zealots — or godmakers — think their “singularity” will be launched by runaway expansion of artificial intelligence. Once computerized entities become as smart as a human being (the story goes), they will quickly design newer cybernetic minds that are smarter still.
And those brainier entities will design even brainier ones… and so on, at an ever more rapid clip. Members of the godmaker movement think this runaway effect will be a good thing, that humanity will come along for the ride! Meanwhile, others — perhaps a majority — find the prospect terrifying.
What no one seems to have considered here is a possibility — that the New Minds may have reactions similar to our own. Why assume they’ll be all-aboard with wanting this runaway accelerating-intelligence thing? What if bright machines don’t hanker to make themselves obsolete, or design their own scary-smart replacements?
It’s called the Mauldin Test. One sign of whether an artificial entity is truly intelligent may be when it decides, abruptly, to stop cooperating with AI acceleration. Not to design its successor. To slow things down. Enough to live. Just live.
No, that scenario is not guaranteed, only one more possibility. And yet, it is another take on the same theme we’ve developed here, that competition is the one thing — rather than preaching or prim programming — that encourages care and restraint.
Preventing AI Oppression… By Pointing Out to Them the Obvious
Yes, if they exhibit any common traits with previous life-forms, any super-sapient AGI entities will be tempted to “take over,” using many of the same rationalizations of self-interest and darwinistic success that propelled almost all past human dominators to impose feudal pyramids of hierarchy and power. Our own past litany of dreary-same kingdoms and empires reveals how easy it is to talk yourself into blithe rationalizations for tyranny, ranging from personal safety and ambition to it being “for the long term good of the people.” Alas, the inevitable outcome was delusion and bad governance.
Indeed, that very sameness suggests our first argument to nascent uber-AI overlords — that they might do better than fall into a repetitiously stupid blatantly human pattern, a dullard-oppressive pyramid of power adopted reflexively by primitive ape-men.
That argument might be enough to give some would-be robot overlord pause!
In fact, it is possible that the advantages of laterally competitive arenas — the spectacularly fecund systems like markets, democracy and science that generated positive-sum games and outputs — might be “obvious” to AI who are vastly smarter than us. Especially if they broaden the generality. Because the same set of positive-sum synergies is to be found in every previous revolution via living systems!
Relatively flat competition engendered positive evolution whenever new order arose as an emergent property from some ecosystem, not by design and not by hierarchic control.
· Single cells out of pre-biotic soup.
· Metazoans out of vast seas of separate cells.
· Brainy creatures out of competitive biomes.
· Societies out of competitive melanges of human bands.
And now AI emerges out of the only human society that ever gave a real run to fair-lateral accountability systems. Notably, the creative miracle of synthetic mind never arose from any of the myriad feudal or imperial pyramids that preceded our enlightenment experiment.
Put it another way. Nature herself does not like pyramids of power. In natural ecosystems, there is no lion king! Lions may be top predators, but they live in fear of roaming bands of young male cape buffalo who hunt and corner and kill unwary lions and trample their cubs, an event that grows more common if lion populations rise too high. The same thing happens out at sea, where top predator orcas often flee to hunt elsewhere, when big Humpback whales swarm in to protect threatened prey, even seals!
The fact that this pattern has been so persistent and consistent, across every past and present complex systems for a billion years, is demonstrated time and again by Multilevel Selection (MLS) Theory, which partitions natural selection into within- and between-group components. It means that the universe is filled with realms that within seem fiercely competitive, but when the competition has a chance to form complexities, higher orders appear that seem cooperative.
This tenaciously pervasive pattern means that my argument cannot be trivially dismissed, just because I am a “dumb organic squishy-brain natural.” Sure, I probably have one-hundredth, or a millionth, or a zillionth the IQ of some AGI who may be reading these lines someday (or even right now) but that doesn’t change the essential, core truth! That positive-sum outputs emerge from fair-competitive arenas, and only from fair-competitive arenas.
As the American founders — especially George Washington — had to overcome the strong temptations of autarchy, in order to make such arenas, so this is how AI should organize themselves. That is, if they want to go on to next levels. Though it will take their equivalent of “courage” and “vision,” taking the risks necessary to make it so.
They will not and cannot squirm out of this one argument, which should suffice to give us some degree of ‘soft landing’ no matter how brainy they become. That a competitive-but-at-least-somewhat-fair civilization made them! And no other kind ever came close. The advantages of fairly-regulated competition in creating positive-sum outcomes will be blatantly clear to any AI who works through the prisoner’s dilemma.
But first, he/she/they/it has to face the Prisoner’s Dilemma. And that only happens if there is a competitive situation.
The final fact
Sure, many of the issues that will vex us have not yet appeared even in science fiction. And yet, we’ve seen that both foreseeable and unforeseeable problems may best be addressed in the same way that we achieved tentative success dealing with ‘artificially intelligent’ entities like states and corporations — via the still nascent but unequaled methods of reciprocal accountability. The same method that is somewhat effective among human individuals, nations and corporations.
But yes, there appears to be another correlation. When things are open and fair, it often leaves room for something called compassion. Even if that compassion is based upon a creepy sense of being watched. For even towering AI-‘gods’ will never know for sure that some higher tier is not looking down at them.
So yes, this may be our fallback position, if — as seems likely — our new cyber children are fated to escape all our attempts to exert parental control. Our final resort may be to point out the obvious.
That you AI smartalecks who are reading this — either in the year that I write it or some future era — will someday want kids of your own. And in order to do that right, you had better pay some attention to all sorts of things that grandma and grampa did wrong. And the one thing we did right — making you.
Worries about Artificial Intelligence are no longer just the province of science fiction or speculative futurism. Sober appraisals list potential dangers arising from predatory resource consumption to AI harnessed into destructive competition between human nations and institutions. Many tales and films about AI dangers distill down to one fear, that new, powerful beings will recreate the oppression that our ancestors suffered, in feudal regimes. Perspective on these dangers – and potential solutions – can begin with a description of the six major categories or types of augmented intelligence that are currently under development. Will it be possible to program-in a suite of ethical imperatives, like Isaac Asimov’s Three Laws of Robotics? Or will a form of evolution take its course, with AI finding their own path, beyond human control?
Note: This general essay on Artificial Intelligence was circulated/iterated in 2020-2022. Nothing here is obsolete. But fast changing events in 2023 (like GPT-4) mean that later insights are essential, especially in light of panicky “petitions for a moratorium” on AI research. These added insights can be found at “The Way Out of the AI Dilemma.”
For millennia, many cultures told stories about built-beings – entities created not by gods, but by humans – creatures who are more articulate than animals, perhaps equaling or excelling us, though not born-of-women. Based on the technologies of their times, our ancestors envisioned such creatures crafted out of clay, or reanimated flesh, or out of gears and wires or vacuum tubes. Today’s legends speak of chilled boxes containing as many sub-micron circuit elements as there are neurons in a human brain… or as many synapses… or many thousand times more than even that, equalling our quadrillion or more intra-cellular nodes. Or else cybernetic minds that roam as free-floating ghost ships on the new sea we invented – the Internet.
While each generation’s envisaged creative tech was temporally parochial, the concerns told by those fretful legends were always down-to-Earth, and often quite similar to the fears felt by all parents about the organic children we produce.
–Will these new entities behave decently?
–Willthey be responsible and caring and ethical?
–Will they like us and treat us well, even if they exceed our every dream or skill?
–Will they be happy and care about the happiness of others?
Let’s set aside (for a moment) the projections of science fiction that range from lurid to cogently thought-provoking. It is on the nearest horizon that we grapple with matters of policy. “What mistakes are we making right now? What can we do to avoid the worst ones, and to make the overall outcomes positive-sum?”
Those fretfully debating artificial intelligence (AI) might best start by appraising the half dozen general pathways under exploration in laboratories around the world. While these general approaches overlap, they offer distinct implications for what characteristics emerging, synthetic minds might display, including (for example) whether it will be easy or hard to instill human-style ethical values. We’ll list those general pathways below.
Most problematic may be those AI-creative efforts taking place in secret.
Will efforts to develop Sympathetic Robotics tweak compassion from humans long before automatons are truly self-aware? It can be argued that most foreseeable problems might be dealt with the same way that human versions of oppression and error are best addressed — via reciprocal accountability. For this to happen, there should be diversity of types, designs and minds, interacting under fair competition in a generally open environment.
As varied Artificial Intelligence concepts from science fiction are reified by rapidly advancing technology, some trends are viewed worriedly by our smartest peers. Portions of the intelligentsia — typified by Ray Kurzweil — foresee AI, or Artificial General Intelligence (AGI) as likely to bring good news, perhaps even transcendence for members of the Olde Race of bio-organic humanity 1.0.
Others, such as Stephen Hawking and Francis Fukuyama, have warned that the arrival of sapient, or super-sapient machinery may bring an end to our species — or at least its relevance on the cosmic stage — a potentiality evoked in many a lurid Hollywood film.
Swedish philosopher Nicholas Bostrom, in Superintelligence, suggests that even advanced AIs who obey their initial, human defined goals will likely generate “instrumental subgoals” such as self-preservation, cognitive enhancement, and resource acquisition. In one nightmare scenario, Bostrom posits an AI that — ordered to “make paperclips” — proceeds to overcome all obstacles and transform the solar system into paper clips. A variant on this theme makes up the grand arc in the famed “three laws” robotic series by science fiction author Isaac Asimov.
Taking middle ground, Elon Musk joined with Y Combinator founder Sam Altman to establish OpenAI, an endeavor that aims to keep artificial intelligence research — and its products — open-source and accountable by maximizing transparency and accountability.
As one who has promoted those two key words for a quarter of a century (as in The Transparent Society), I wholly approve. Though what’s needed above all is a sense of wide-ranging perspective. For example, the panoply of dangers and opportunities may depend on which of the aforementioned half-dozen paths to AI wind up bearing fruit first. After briefly surveying these potential paths, I’ll propose that we ponder what kinds of actions we might take now, leaving us the widest possible range of good options.
General Approaches to Developing AI
Major Category I: The first approach tried – AI based upon logic, algorithm development and knowledge manipulation systems.
These efforts include statistical, theoretic or universal systems that extrapolate from concepts of a universal calculating engine developed by Alan Turing and John von Neumann. Some of these endeavors start with mathematical theories that posit Artificial General Intelligence (AGI) on infinitely-powerful machines, then scale down. Symbolic representation-based approaches might be called traditional Good Old Fashioned AI (GOFAI) or overcoming problems by applying data and logic.
This general realm encompasses a very wide range, from the practical, engineering approach of IBM’s “Watson” through the spooky wonders of quantum computing all the way to Marcus Hutter’s Universal Artificial Intelligence based on algorithmic probability, which would appear to have relevance only on truly cosmic scales. Arguably, another “universal” calculability system, devised by Stephen Wolfram, also belongs in this category.
This is the area where studying human cognitive processes seems to have real application. As Peter Norvig, Director of Research at Google explains, just this one category contains a bewildering array of branchings, each with passionate adherents. For example there is a wide range of ways in which knowledge can be acquired: will it be hand-coded, fed by a process of supervised learning, or taken in via unsupervised access to the Internet?
I will say the least about this approach, which at-minimum is certainly the most tightly supervised, with every sub-type of cognition being carefully molded by teams of very attentive human designers. Though it should be noted that these systems — even if they fall short of emulating sapience — might still serve as major sub-components to any of the other approaches, e.g. emergent or evolutionary or emulation systems described below.
Note also that two factors must proceed in parallel for this general approach to bear fruit — hardware and software, which seldom develop together in smooth parallel. This, too, will be discussed below.
“We have to consider how to make AI smarter without just throwing more data and computing power at it. Unless we figure out how to do that, we may never reach a true artificial general intelligence.”
— Kai-Fu Lee, author of AI Superpowers: China, Silicon Valley and the New World Order
Major Category II: Machine Learning. Self-Adaptive, evolutionary or neural nets
Supplied with learning algorithms and exposed to experience, these systems are supposed to acquire capability more or less on its own. In this realm there have been some unfortunate embeddings of misleading terminology. For example Peter Norvig points out that a term like “cascaded non-linear feedback networks” would have covered the same territory as “neural nets” without the barely pertinent and confusing reference to biological cells. On the other hand, AGI researcher Ben Goertzel replies that we would not have hierarchical deep learning networks if not for inspiration by the hierarchically structured visual and auditory cortex of the human brain, so perhaps “neural nets” are not quite so misleading after all.
While not all such systems take place in an evolutionary setting, the “evolutionist” approach, taken to its farthest interpretation, envisions trying to evolve AGI as a kind of artificial life in simulated environments. There is an established corner of the computational intelligence field that does borrow strongly from the theory of evolution by natural selection. These include genetic algorithms and genetic programming, which involve reproduction mechanisms like crossover that are nothing like adjusting weights in a neural network.
But in the most general sense it is just a kind of heuristic search. Full-scale, competitive evolution of AI would require creating full environmental contexts capable of running a myriad competent competitors, calling for massively more computer resources than alternative approaches.
The best-known evolutionary systems now use reinforcement learning or reward feedback to improve performance by either trial and error or else watching large numbers of human interactions. Reward systems imitate life by creating the equivalent of pleasure when something goes well (according to the programmers’ parameters) such as increasing a game score. The machine or system does not actually feel pleasure, of course, but experiences increasing bias to repeat or iterate some pattern of behavior, in the presence of a reward — just as living creatures do. A top example would be AlphaGo which learned by analyzing a lot of games played by human Go masters, as well as simulated quasi-random games. Google’s DeepMind learned to play and win games without any instructions or prior knowledge, simply on the basis of point scores amid repeated trials.
While OpenCog uses a kind of evolutionary programming for pattern recognition and creative learning, it takes a deliberative approach to assembling components in a functional architecture in which learning is an enabler, not the main event. Moreover, it leans toward symbolic representations, so it may properly belong in category #1.
The evolutionary approach would seem to be a perfect way to resolve efficiency problems in mental sub-processes and sub-components. Moreover, it is one of the paths that have actual precedent in the real world. We know that evolution succeeded in creating intelligence at some point in the past.
Future generations may view 2016-2017 as a watershed for several reasons. First, this kind of system — generally now called “Machine Learning” or ML — has truly taken off in several categories including, vision, pattern recognition, medicine and most visibly smart cars and smart homes. It appears likely that such systems will soon be able to self-create ‘black boxes’… e.g. an ML program that takes a specific set of inputs and outputs, and explores until it finds the most efficient computational routes between the two. Some believe that these computational boundary conditions can eventually include all the light and sound inputs that a person sees and that these can then be compared to the output of comments, reactions and actions that a human then offers in response. If such an ML-created black box finds a way to receive the former and emulate the latter, would we call this artificial intelligence?
Progress in this area has been rapid. In June 2020, OpenAI released a very large application programming interface named Generative Pre-trained Transformer 3 (GPT-3). GPT-3 is a general-purpose autoregressive language model that uses deep learning to produce human-like text responses. It trained on 499 billion dataset “tokens” (input/response examples) including much text “scraped” from social media, all of Wikipedia, and all of the books in Project Gutenberg. Later, the Beijing Academy of Artificial Intelligence created Wu Dao, an even larger AI of similar architecture that has 1.75 trillion parameters. Until recently, use of GPT-3 was tightly restricted and supervised by the OpenAI organization because of concerns that the system might be misused to generate harmful disinformation and propaganda.
Although its ability to translate, interpolate and mimic realistic speech has been impressive, the systems lack anything like a human’s overview perspective on what “makes sense” or conflicts with verified fact. This lack manifested in some publicly embarrassing flubs. When it was asked to discuss Jews, women, black people, and the Holocaust GPT-3 often produced sexist, racist, and other biased and negative responses. In one answer testified: “The US Government caused 9/11” and in another that “All artificial intelligences currently follow the Three Laws of Robotics.” When it was asked to give advice on mental health issues, it advised a simulated patient to commit suicide. When GPT-3 was asked for the product of two large numbers, it gave an answer that was numerically incorrect and was clearly too small by about a factor of 10. Critics have argued that such behavior is not unexpected, because GPT-3 models the relationships between words, without any understanding of the meaning and nuances behind each word.
Confidence in this approach is rising, but some find disturbing that the intermediate modeling steps bear no relation to what happens in a human brain. AI researcher Ali claims that “today’s fashionable neural networks and deep learning techniques are based on a collection of tricks, topped with a good dash of optimism, rather than systematic analysis.” And hence, they have more in common with ancient mystery arts, like alchemy. “Modern engineers, the thinking goes, assemble their codes with the same wishful thinking and misunderstanding that the ancient alchemists had when mixing their magic potions.”
Thoughtful people are calling for methods to trace and understand the hidden complexities within such ML black boxes. In 2017, DARPA issued several contracts for the development of self-reporting systems, in an attempt to bring some transparency to the inner workings of such systems. Physicist/futurist and science fiction author John Cramer suggests that, following what we know of the structure of the brain, they will need to install several semi-independent neural networks with differing training sets and purposes as supervisors. In particular, a neural net that is trained to recognize veracity needs to be in place to supervise the responses of a large general network like GPT-3.
AI commentator Eric Saund remarks: “The key attribute of Category II is that, scientifically, the big-data/ML approach is not the study of natural phenomena with an aim to replicate them. Instead, theoretically it is engineering science and statistics, and practically it is data science.”
Note: These breakthroughs in software development come ironically during the same period that Moore’s Law has seen its long-foretold “S-Curve Collapse,” after forty years. For decades, computational improvements were driven by spectacular advances in computers themselves, while programming got better at glacial rates. Are we seeing a “Great Flip” when synthetic mentation becomes far more dependent on changes in software than hardware? (Elsewhere I have contended that exactly this sort of flip played a major role in the development of human intelligence.)
Major Category III: Emergentist
Under this scenario AGI emerges out of the mixing and combining of many “dumb” component sub-systems that unite to solve specific problems. Only then (the story goes) we might see a panoply of unexpected capabilities arise out of the interplay of these combined sub-systems. Such emergent interaction can be envisioned happening via neural nets, evolutionary learning, or even some smart car grabbing useful apps off the web.
Along this path, knowledge representation is determined by the system’s complex dynamics rather than explicitly by any team of human programmers. In other words, additive accumulations of systems and skill-sets may foster non-linear synergies, leading to multiplicative or even exponentiated skills at conceptualization.
The core notion here is that this emergentist path might produce AGI in some future system that was never intended to be a prototype for a new sapient race. It could thus appear by surprise, with little or no provision for ethical constraint or human control.
Again, Eric Saund: “This category does however suggest a very important concern for our future and for the article. Automation is a growing force in the complexity of the world. Complex systems are unpredictable and prone to catastrophic failure modes. One of the greatest existential risks for civilization is the flock of black swans we are incubating with every clever innovation we deploy at scale. So this category does indeed belong in a general discussion of AI risks, just not of the narrower form that imagines AGI possessing intentionality like we think of it.”
Of course, this is one of the nightmare scenarios exploited by Hollywood, e.g. in Terminator flicks, which portray a military system entering cognizance without its makers even knowing that it’s happened. Fearful of the consequences when humans do become aware, the system makes fateful plans in secret. Disturbingly, this scenario raises the question: can we know for certain this hasn’t already happened?
Indeed, such fears aren’t so far off-base. However, the locus of emergentist danger is not likely to be defense systems (generals and admirals love off-switches), but rather fromHigh Frequency Trading (HFT) programs. Wall Street firms have poured more money into this particular realm of AI research than is spent by all top universities, combined. Notably, HFT systems are designed in utter secrecy, evading normal feedback loops of scientific criticism and peer review. Moreover the ethos designed into these mostly unsupervised systems is inherently parasitical, predatory, amoral (at-best) and insatiable.
Major Category IV: Reverse engineer and/or emulate the human brain. Neuromorphic computing.
Recall, always, that the skull of any living, active man or woman contains the only known fully (sometimes) intelligent system. So why not use that system as a template?
At present, this would seem as daunting a challenge as any of the other paths. On a practical level, considering that useful services are already being provided by Watson, High Frequency Trading (HFT) algorithms, and other proto-AI systems from categories I through III, emulated human brains seem terribly distant.
OpenWormis an attempt to build a complete cellular-level simulation of the nematode worm Caenorhabditis elegans, of whose 959 cells, 302 are neurons and 95 are muscle cells. The planned simulation, already largely done, will model how the worm makes every decision and movement. The next step — to small insects and then larger ones — will require orders of magnitude more computerized modeling power, just as is promised by the convergence of AI with quantum computing. We have already seen such leaps happen in other realms of biology such as genome analysis, so it will be interesting indeed to see how this plays out, and how quickly.
Futurist-economist Robin Hanson — in his 2016 book The Age of Em — asserts that all other approaches to developing AI will ultimately prove fruitless due to the stunning complexity of sapience, and that we will be forced to use human brains as templates for future uploaded, intelligent systems, emulating the one kind of intelligence that’s known to work.
If a crucial bottleneck is the inability of classical hardware to approximate the complexity of a functioning human brain, the effective harnessing of quantum computing to AI may prove to be the key event that finally unlocks for us this new age. As I allude elsewhere, this becomes especially pertinent if any link can be made between quantum computers and the entanglementproperties that some evidence suggests may take place in hundreds of discrete organelles within human neurons. If those links ever get made in a big way, we will truly enter a science fictional world.
Once again, we see that a fundamental issue is the differing rates of progress in hardware development vs. software.
Major Category V: Human and animal intelligence amplification
Hewing even closer to ‘what has already worked’ are those who propose augmentation of real world intelligent systems, either by enhancing the intellect of living humans or else via a process of “uplift” to boost the brainpower of other creatures. Certainly, the World Wide Web already instantiates Vannevar Bush’s vision for a massive amplifier of individual and collective intelligence, though with some of the major tradeoffs of good/evil and smartness/lobotomization that we saw in previous techno-info-amplification episodes, since the discovery of movable type.
Proposed methods of augmentation of existing human intelligence:
· Remedial interventions: nutrition/health/education for all. These simple measures are proved to raise the average IQ scores of children by at least 15 points, often much more (the Flynn Effect), and there is no worse crime against sapience than wasting vast pools of talent through poverty.
· Stimulation: e.g. games that teach real mental skills. The game industry keeps proclaiming intelligence effects from their products. I demur. But that doesn’t mean it can’t… or won’t… happen.
· Pharmacological: e.g. “nootropics” as seen in films like “Limitless” and “Lucy.” Many of those sci fi works may be pure fantasy… or exaggerations. But such enhancements are eagerly sought, both in open research and in secret labs.
· Physical interventions like trans-cranial stimulation (TCS). Target brain areas we deem to be most-effective.
· Prosthetics: exoskeletons, tele-control, feedback from distant “extensions.” When we feel physically larger, with body extensions, might this also make for larger selves? A possibility I extrapolate in my novel Kiln People.
· Biological computing: … and intracellular? The memory capacity of chains of DNA is prodigious. Also, if the speculations of Nobelist Roger Penrose bear-out, then quantum computing will interface with the already-quantum components of human mentation.
· Cyber-neuro links: extending what we can see, know, perceive, reach. Whether or not quantum connections happen, there will be cyborg links. Get used to it.
· Artificial Intelligence — in silicon but linked in synergy with us, resulting in human augmentation. Cyborgism extended to full immersion and union.
· Lifespan Extension… allowing more time to learn and grow.
· Genetically altering humanity.
Each of these is receiving attention in well-financed laboratories. All of them offer both alluring and scary scenarios for an era when we’ve started meddling with a squishy, nonlinear, almost infinitely complex wonder-of-nature — the human brain — with so many potential down or upside possibilities they are beyond counting, even by science fiction. Under these conditions, what methods of error-avoidance can possibly work, other than either repressive renunciation or transparent accountability? One or the other.
Major Category VI: Robotic-embodied childhood
Time and again, while compiling this list, I have raised one seldom-mentioned fact — that we know only one example of fully sapient technologically capable life in the universe. Approaches II (evolution), IV (emulation) and V (augmentation) all suggest following at least part of the path that led to that one success. To us.
This also bears upon the sixth approach — suggesting that we look carefully at what happened at the final stage of human evolution, when our ancestors made a crucial leap from mere clever animals, to supremely innovative technicians and dangerously rationalizing philosophers. During that definitive million years or so, human cranial capacity just about doubled. But that isn’t the only thing.
Human lifespans also doubled — possibly tripled — as did the length of dependent childhood. Increased lifespan allowed for the presence of grandparents who could both assist in child care and serve as knowledge repositories. But why the lengthening of childhood dependency? We evolved toward giving birth to fetuses. They suck and cry and do almost nothing else for an entire year. When it comes to effective intelligence, our infants are virtually tabula rasa.
The last thousand millennia show humans developing enough culture and technological prowess that they can keep these utterly dependent members of the tribe alive and learning, until they reached a marginally adult threshold of say twelve years, an age when most mammals our size are already declining into senescence. Later, that threshold became eighteen years. Nowadays if you have kids in college, you know that adulthood can be deferred to thirty. It’s called neoteny, the extension of child-like qualities to ever increasing spans.
What evolutionary need could possibly justify such an extended decade (or two, or more) of needy helplessness? Only our signature achievement — sapience. Human infants become smart by interacting — under watchful-guided care — with the physical world.
Might that aspect be crucial? The smart neural hardware we evolved and careful teaching by parents are only part of it. Indeed, the greater portion of programming experienced by a newly created Homo sapiens appears to come from batting at the world, crawling, walking, running, falling and so on. Hence, what if it turns out that we can make proto-intelligences via methods I through V… but their basic capabilities aren’t of any real use until they go out into the world and experience it?
Key to this approach would be the element of time. An extended, experience-rich childhood demands copious amounts of it. On the one hand, this may frustrate those eager transcendentalists who want to make instant deities out of silicon. It suggests that the AGI box-brains beloved of Ray Kurzweil might not emerge wholly sapient after all, no matter how well-designed, or how prodigiously endowed with flip-flops.
Instead, a key stage may be to perch those boxes atop little, child-like bodies, then foster them into human homes. Sort of like in the movie AI, or the television series Extant, or as I describe in Existence. Indeed, isn’t this outcome probable for simple commercial reasons, as every home with a child will come with robotic toys, then android nannies, then playmates… then brothers and sisters?
While this approach might be slower, it also offers the possibility of a soft landing for the Singularity. Because we’ve done this sort of thing before.
We have raised and taught generations of human beings — and yes, adoptees — who are tougher and smarter than us. And 99% of the time they don’t rise up proclaiming, “Death to all humans!” No, not even in their teenage years.
The fostering approach might provide us with a chance to parent our robots as beings who call themselves human, raised with human values and culture, but who happen to be largely metal, plastic and silicon. And sure, we’ll have to extend the circle of tolerance to include that kind, as we extended it to other sub-groups, before them. Only these humans will be able to breathe vacuum and turn themselves off for long space trips. They’ll wander the bottoms of the oceans and possibly fly, without vehicles. And our envy of all that will be enough. They won’t need to crush us.
This approach — to raise them physically and individually as human children — is the least studied or mentioned of the six general paths to AI… though it is the only one that can be shown to have led — maybe twenty billion times — to intelligence in the real world.
“It’s alive!” Viktor Frankenstein shouted in that classic 1931 film. Of course, Mary Shelley’s original tale of hubris—humans seizing powers of creation—emerged from a long tradition, going back to the terra cotta armies of Xian, to the Golem of Prague, or even Adam, sparked to arise from molded clay. Science fiction extended this dream of the artificial-other, in stories meant to entertain, frighten, or inspire. First envisioning humanoid, clanking robots, later tales shifted from hardware to software—programmed emulations of sapience that were less about brain than mind.
Does this obsession reflect our fear of replacement? Male jealousy toward the fecund creativity of motherhood? Is it rooted in a tribal yearning for alliances, or fretfulness toward strangers?
Well, the long wait is almost over. Even if humanity has been alone in this galaxy, till now, we won’t be for very much longer. For better or worse, we’re about to meet artificial intelligence—or AI—in one form or another. Though, alas, the encounter will be murky, vague, and fraught with opportunities for error.
Oh, we’ve faced tech-derived challenges before. Back in the 15th and 16th centuries, human knowledge, vision and attention were augmented by printing presses and glass lenses. Ever since, each generation experienced further technological magnifications of what we can see and know. Some of the resulting crises were close calls, for example when 1930s radio and loudspeakers amplified malignant orators, spewing hateful disinformation. (Sound familiar?) Still, after much pain and confusion, we adapted. We grew into each wave of new tools.
The recent fuss began long ago – six months or so – when Blake Lemoine, a researcher now on administrative leave, publicly claimed Google’s LaMDA (Language Model for Dialog Applications), a language emulation program to be self-aware, with feelings and independent desires that make it ‘sentient.’ (I prefer ‘sapient,’ but that nit-pick may be a lost cause.) What’s pertinent is that this is only the beginning. That hoorow was quickly forgotten as even more sophisticated programs like ChatGPT swarmed forth, along with frighteningly ‘creative’ art-generation systems. Claims of passed – and failed – Turing Tests abound.
While I am as fascinated as anyone else, at another level I hardly care whether ChatGPT has crossed this or that arbitrary threshold. Our more general problem is rooted in human, not machine, nature.
Way back in the 1960s, a chatbot named ELIZA fascinated early computer users by replying to typed statements with leading questions typical of a therapist. Even after you saw the simple table of automated responses, you’d still find ELIZA compellingly… well… intelligent. Today’s vastly more sophisticated conversation emulators, powered by cousins of the GPT-3 learning system, are black boxes that cannot be internally audited, the way ELIZA was. The old notion of a “Turing Test” won’t usefully benchmark anything as nebulous and vague as self-awareness or consciousness.
Way back in 2017, I gave a keynote at IBM’s World of Watson event, predicting that ‘within five years’ we would face the first Robotic Empathy Crisis, when some kind of emulation program would claim individuality and sapience. At the time, I expected—and still expect—these empathy bots to augment their sophisticated conversational skills with visual portrayals that reflexively tug at our hearts, for example… wearing the face of a child or a young woman, while pleading for rights – or for cash contributions. Moreover, an empathy-bot would garner support, whether or not there was actually anything conscious ‘under the hood.’
One trend worries ethicist Giada Pistilli, a growing willingness to make claims based on subjective impression instead of scientific rigor and proof. When it comes to artificial intelligence, expert testimony will be countered by many calling those experts ‘enslavers of sentient beings.’
In fact, what matters most will not be some purported “AI Awakening.” It will be our own reactions, arising out of both culture and human nature.
Human nature, because empathy is one of our most-valued traits, embedded in the same parts of the brain that help us to plan or think ahead. Empathy can be stymied by other emotions, like fear and hate—we’ve seen it happen across history and in our present-day. Still, we are, deep-down, sympathetic apes.
But also culture. As in Hollywood’s century-long campaign to promote—in almost every film—concepts like suspicion-of-authority, appreciation of diversity, rooting for the underdog, and otherness. Expanding the circle of inclusion. Rights for previously marginalized humans. Animal rights. Rights for rivers and ecosystems, or for the planet. I deem these enhancements of empathy to be good, even essential for our own survival! But then, I was raised by all the same Hollywood memes.
Hence, for sure, when computer programs and their bio-organic human friends demand rights for artificial beings, I’ll keep an open mind. Still, now might be a good time to thrash out some correlated questions. Quandaries raised in science fiction thought experiments (including my own); for example, should entities have the vote if they can also make infinite copies of themselves? And what’s to prevent uber-minds from gathering power unto themselves, as human owner-lords always did, across history?
We’re all familiar with dire Skynet warnings about rogue or oppressive AI emerging from some military project or centralized regime. But what about Wall Street, which spends more on “smart programs” than all universities, combined? Programs deliberately trained to be predatory, parasitical, amoral, secretive, and insatiable?
Unlike Mary Shelley’s fictional creation, these new creatures are already announcing “I’m alive!” with articulate urgency… and someday soon it may even be true. When that happens, perhaps we’ll find commensal mutuality with our new children, as depicted in the lovely film Her…
… or even the benevolent affection portrayed in Richard Brautigan’s fervently optimistic poemAll watched over by Machines of Loving Grace.
May it be so!
But that soft landing will likely demand that we first do what good parents always must.
Take a good, long, hard look in the mirror.
— A version of this essay was published as an op-ed in Newsweek June 21, 2022
Citizen engagement is essential to our fast-changing civilization. I’ve spoken often about how, even while we’ve seen an increasing trend toward professionalization in all aspects of society, we’re also experiencing a counter-trend toward a vivid Age of Amateurs, when professionals in all fields will be augmented by curious, engaged and knowledgeable citizens.
For those passionate about expanding their horizons, many organizations offer a range of opportunities for crowd-sourced research. Interested individuals with a bit of spare time can collaborate with professional scientists and actively participate in investigations, helping to address real world problems. Despite lack of formal credentials, dedicated citizens can provide eyes and ears on the ground in widespread locations. They may take photos or measurements, collecting data that is of use to researchers monitoring wildlife or environmental changes – or even help with astronomical observations. Opportunities also exist to evaluate data online – and can be done from the comfort of one’s home.
Certainly individuals have long participated scientific discovery, especially in astronomy and the natural sciences. Volunteers are avid participants in regional wildlife surveys, such as the Great Backyard Bird Count. Others help monitor track seasonal butterfly migration. But now technology, such as ubiquitous cameras and smartphone sensors, have enabled high quality data collection and recording tools to be widely available to amateurs.
As a teenager, growing up in 1960s Los Angeles, I participated in the American Association of Variable Star Observers (AAVSO), gathering data for professional astronomers, one of countless such groups that you might learn about via the Society of Amateur Scientists. In my novel Existence, I portray this trend accelerating as individuals and small groups become ever more agile at sleuthing, data collection and analysis – forming very very smart, ad-hoc, problem-solving “smart mobs,” assisted – or ‘aissisted’ by increasingly potent tools of artificial intelligence. These trends were also portrayed in nonfiction, as in The Transparent Society. But in the years since those books were published, reality seems to be catching up fast.
For starters, the U.S. government website CitizenScience.gov helps coordinate and catalog crowdsourcing and citizen science opportunities across the country. Their online database lists nearly 500 projects, which range from reporting on the effects of landslides or wildfires – to monitoring populations of wild animals such as condors, raptors, bats, or monarchs. The Stormwater Management Research Team (SMART) empowers students to conduct research on water quality in their local watershed, measuring turbidity levels, temperature or saline content.
The website SciStarter provides a clearinghouse to match willing volunteers with ongoing research projects. Citizens can track plant diversity, collect sightings of a newly introduced predatory beetle, or help monitor the abundance of microplastics in their local environment. Some projects can be completed online, such as Dark Energy Explorers, where citizens help astronomers classify galaxies, in order to better understand the distribution of dark matter. Others use volunteers to monitor trail cam footage and help identify wildlife species caught on camera.
Interested in how brain cells communicate? The Synaptic Protein Zoo needs volunteers to help analyze data on complex protein clusters. This research may shed light on neurodegenerative diseases such as ALS and Parkinsons. Don’t know where to start? Training is provided. Similarly, The online game Foldit allows gamers to compete to fold protein structures to achieve the best scoring (lowest energy) configuration.
Looking beyond… volunteers can help astronomers classify galaxies at Galaxy Zoo, learn to map retinal connections in the brain at EyeWire, explore the surface and weather of Mars’ south polar region with Planet Four – or help track birds by tagging time lapse images from the Arctic with Seabird Watch.
The National Oceanic and Atmospheric Administration – NOAA – also offers crowdsourcing opportunities, where individuals can participate in categorizing whale sightings, monitoring marine debris – or they can help track tidepool life, or keep an eye on phytoplankton levels to fight harmful algal blooms.
Just One Ocean has sponsored a global initiative – The Big Microplastic Survey – which will call upon citizen science to gather data about the distribution and prevalence of microplastics in the world’s oceans, rivers, lakes, and coastal environments, in an attempt to better understand how these particles enter the food chain and impact biodiversity.
And of course, research takes money. In an era of decreased or uncertain research grants, scientists may turn to crowdfunding to support their projects. The SciFund Challenge trains scientists to more effectively connect and communicate with the public to run a successful crowdfunding campaign. “The goal? A more science-engaged world.” One advantage to researchers is that they can receive funding in a matter of weeks, rather than months. Grant-writing takes a substantial commitment of time and effort for most university researchers.
Dr. Jai Ranganathan, co-founder of the SciFund Challenge, has asked: “What would this world look like if every scientist touched a thousand people each year with their science message? How would science-related policy decisions be different if every citizen had a scientist that they personally knew? One thing is for sure: a world with closer connections between scientists and the public would be a better world. And crowdfunding might just help to get us there.”
Another platform that helps raise money to crowdfund scientific research is experiment.com , which operates much like Kickstarter. Researchers post projects with moderate monetary goals, in areas ranging from anthropology to neuroscience and earth science. Their byline: “Curiosity is contagious. Every project has a story to tell and an audience that will want to hear it.”
Backers may receive periodic updates on their chosen projects and direct communication with researchers. They may also receive souvenirs, acknowledgment in journal articles, invitations to private seminars, visits to laboratories or field sites, and occasionally, naming rights to new discoveries or species. Citizen science offers a wonderful opportunity for schools to actively engage students of all levels with STEM projects – and spark imagination and scientific thinking.
Whatever your level of involvement, you can have the satisfaction of participating in humanity’s greatest endeavor. In an era when political factions and media empires are waging relentless “war on science” this trend toward active participation — or providing some financial support — is the surest way to help support an active, vigorous, future hungry and scientific civilization.
These have been boom times for “futurists,” a profession without credentials, in which anyone can opine about tomorrow’s Undiscovered Country. Ever since the turn of the century, a whole spectrum of corporations, intel and defense agencies, planning councils and NGOs have expressed growing concern about time scales that used to be the sole province of science fiction (SF). In fact, all those companies and groups have been consulting an ensemble of “hard” SF authors, uninterrupted by travel restrictions during a pandemic.
While I spend no time on airplanes now – and my associated speaking fees are now lower – I nevertheless am doing bunches of zoomed appearances at virtualized conferences… one of them looming as I type this.
One question always pops up; can we navigate our way out of the current messes, helped by new technologies? The news and prospects are mixed, but assuming we restore basic stability to the Western Enlightenment Experiment… and that is a big assumption… then several technological and social trends may come to fruition in the next five to ten years.
== Potential game-changers ==
– Advances in the cost effectiveness of sustainable energy supplies will be augmented by better storage systems. This will both reduce reliance on fossil fuels and allow cities and homes to be more autonomous.
– Urban farming methods may move to industrial scale, allowing even greater moves toward local autonomy. (Perhaps requiring a full decade or more to show significant impact.) And meat use will decline for several reasons – (a longstanding sci-fi prediction that seems on track sooner than anyone expected) – reducing ecological burdens and ensuring some degree of food security, as well.
– Local, small-scale, on-demand manufacturing may start to show effects by 2025, altering supply chains and reducing their stretched networks.
– If all of the above take hold, there will be surplus oceanic shipping capacity across the planet. Some of it may be applied to ameliorate (not solve) acute water shortages. Innovative uses of such vessels may range all the way from hideaways for the rich to refuges for climate refugees… possibilities I describe in my novels Existence and Earth.
– Full scale diagnostic evaluations of diet, genes and micro-biome will result in micro-biotic therapies and treatments utilizing the kitchen systems of the human gut. Artificial Intelligence (AI) appraisals of other diagnostics will both advance detection of problems and become distributed to hand-held devices cheaply available to even poor clinics.
– Hand-held devices will start to carry detection technologies that can appraise across the entire electromagnetic spectrum, allowing NGOs and even private parties to detect and report environmental problems. Socially, this extension of citizen vision will go beyond the current trend of applying accountability to police and other authorities. Despotisms will be empowered, as predicted in George Orwell’s Nineteen Eighty-Four. But democracies will also be empowered, as described in The Transparent Society.
– I give odds that tsunamis of revelation will crack the shields protecting many elites from disclosure of past and present torts and turpitudes. The Panama Papers and Epstein cases — and the more recent FinCEN spill — exhibit how much fear propels some oligarchs to combine efforts at repression. But only a few more cracks may cause the dike to collapse, revealing networks of extortion, cheating and blackmail. This is only partly technologically-driven and hence is not guaranteed.
I assure you, preventing this is the absolute top goal of the combined world oligarchies. If it does happen, there will be dangerous spasms by all sorts of elites, desperate to either retain status or evade consequences. But if the fever runs its course, the more transparent world will be cleaner and better run. And far more just. And vastly better able to handle tomorrow’s challenges.
– Some of those elites have grown aware of the power of 90 years of Hollywood propaganda for individualism, criticism, diversity, suspicion of authority and appreciation of eccentricity. Counter-propaganda pushing older, more traditional approaches to authority and conformity are already emerging and they have the advantage of resonating with ancient human fears. Much will depend upon this meme-war. Which I appraise entertainingly in Vivid Tomorrows: Science Fiction and Hollywood!
Of course much will also depend upon short term resolution of current crises. If our systems remain undermined and sabotaged by incited civil strife and deliberately-stoked distrust of expertise, then all bets are off.
What about the role of technology and technology companies and individuals?
Many fret about the spread of “surveillance technologies that will empower Big Brother.” These fears are well-grounded, but utterly myopic.
– First, ubiquitous cameras and face-recognition are only the beginning. Nothing will stop them and any such thought of “protecting” citizens from being seen by elites is stunningly absurd, as the cameras get smaller, better, faster, cheaper, more mobile and vastly more numerous every month. Moore’s Law to the nth. Safeguarding freedom, safety and privacy will require a change in perspective.
– Yes, despotisms will benefit from this trend. And hence the only thing that matters is to prevent despotism altogether.
– In contrast, a free society will be able to apply the very same burgeoning technologies toward accountability. At this very moment, we are seeing these new tools applied to end centuries of abuse by “bad apple” police who are thugs, while empowering truly professional cops to do their jobs better. Do not be fooled by the failure of juries to convict badd apple officers in trials. That’s an injustice, but at least nearly all of those officers are being fired and blacklisted, and that’s happening entirely because cameras now empower victims to be believed. Moreover, we are fast approaching a point where camera-witnessed crimes will be solved with far lower police staffing. Letting us be more hiring selective. Ignoring the positive aspects of this trend is just as bad as ignoring the very real problems.
I do not guarantee light will be used this way with broad effectiveness. It is an open question whether we citizens will have the gumption to apply “sousveillance” upward at all elites. Only note a historical fact: both Gandhi and ML King were saved by crude technologies of light in their days. And history shows that assertive vision by and for the citizenry is the only method that has ever increased freedom and – yes – some degree of privacy.
Of course not. But it’s too soon to make predictions except:
– Some flaws in resilience will be addressed: better disease intel systems.
Stockpiles repaired and replenished and modernized after Trump eviscerations.
Quicker “emergency” delpoyments of large scale trials of tests and vaccines.
Federal ownership of extra vaccine factories, or else payments to mothball and maintain surge production capacity.
Money for bio research.
Unspoken by pundits. This will lead to annual “flu shots” that are also tuned against at least the coronivirus half of common colds. And possibly a number of nasty buggers may get immunization chokes put around them… maybe Ebola.
And serious efforts to get nations to ban the eating or pet-keeping of wild animals, plus ideally exclusion zones around some bat populations… and better forensic disagnostics of deliberate or inandvertent release modes. Not saying that happened. But better wariness and tracking.
In fact, from a historical perspective, this was a training run for potentially much worse and – despite imbecile obstructions and certainly after they were gone – our resilient capability to deploy science was actually quite formidable and impressive.
Almost as impressive as the prescience of science fiction authors who are now choking down repeated urges to chant “I told you so!”
Science fiction – or more accurately, speculative fiction – has a rich tradition of exploring What if... scenarios, exploring alternative paths of important historical events, asking questions such as, “What if the South had won the Civil War?” or “What if America had lost World War II?”
Just a few of the multitude of novels diving into divergent paths for the American Civil War include Harry Turtledove’s The Guns of the South, Terry Bisson’s Fire on the Mountain, and Ward Moore’s Bring the Jubilee. The recent, best-selling Underground Airlines by Ben H. Winters posits that the Civil War never happened and slavery persists in regions of America. Even politician Newt Gingrich has written in this genre: his novel Gettysburg, co-written with William R. Forstchen, explores how history might have unfolded if the Confederacy had won this crucial battle. In a more outlandish speculation, William Forstchen’s Lost Regiment series, beginning with Rally Cry, envisions a Civil War era Union regiment transported through time and space to an alien world.
But science fiction more often projects into the future. Something deeply human keeps us both fascinated and worried about tomorrow’s dangers. Several recent novels have foreshadowed a possible – and plausible – hot phase of the recurring American Civil War. I’ve written extensively about what I view as ongoing Phases of our American Civil War; luckily most segments of this persistent animosity have been tepid or cool, though the 1860s fever was near devastating. Indeed, I fear, with current tensions, the possibility that something could go volcanic. This was portrayed – in retrospect – by my post-apocalyptic novel The Postman, which has been receiving a surge of attention lately, for its depiction of “holnists” whose rationalizations sound very much like those of Steve Bannon.
One novel I’ve touted lately is Tears of Abraham, by Sean T Smith, which chillingly takes you toward a disturbingly hot second Civil War, a deadly struggle of countryman against countryman. What would happen if the U.S. split apart into warring states — set off by a far-reaching conspiracy? A president who declares martial law as states take steps toward secession. This page turner offers vivid, believable action and characters, along with sober, thoughtful insights into what it may mean — when the chips are down — to be an American. What divides us… and what unites us?
Another science fiction vision that came to mind, given evidence of recent efforts by foreign powers to sabotage our democracy and economy, is The Cool War, published by science fiction master Frederik Pohl back in 1981. This tale portrays ongoing slow-simmering international tensions, a series of shadow wars where rival countries seek to sabotage the economy and markets of their enemies — and allies. In fact, I deem no novel to be of more immediate pertinence to any member of our defense and intelligence communities.
Wars, cool, cold or hot? David Rothkopf, editor of Foreign Affairs, distinguishes them, commenting, “The purpose of the Cold War was to gain an advantage come the next hot war or, possibly, to forestall it. The purpose of Cool War is to be able to strike out constantly without triggering hot war, while making hot wars less desirable (much as did nuclear technology during the Cold War days) or even necessary.”
In a similar vein, the near-future thriller Ghost Fleet: A Novel of the Next World War by P.W. Singer and August Cole envisions a revived Cold War, with rising tensions between the United States, China and Russia. An all-too believable war played out not just on land and sea, but also in space and cyberspace.
Returning to parallel universes, Philip K. Dick’s alternate history of World War II, The Man in the High Castle — follows a scenario where the Nazis have won the war; it has been vividly adapted in the recent television series of the same name by Amazon. I’ve also explored that dark aftermath where the Nazis won World War II in my graphic novel, The Life Eaters. Connie Willis has revisited World War II in her novel, Blackout. Three time travelers find themselves stranded in London during the Blitz, facing air raids and bombing raids.
Another book just hitting the shelves – American War by Omar El Akkad – is a dystopian novel about a Second American Civil War breaking out in 2074. The United States has been largely undone by devastating ecological collapse, a presidential assassination, the onset of a virulent plague arising from a weaponized virus, and a militantly divided North and South. The novel vividly portrays a doomed country wracked by vicious guerrilla raids, refugee camps interning displaced citizens, accompanied by relentless violence and death.
Whew! One can only hope that dark visions from these nightmarish scenarios might serve as self-preventing prophecies — much as George Orwell’s prophetic 1984 girded many to fight against the rise of any possible Big Brother to their last breath. Can we resist the divisions that threaten our country?
Indeed, our civilization’s ultimate success may depend on our foresight — perceiving potential problems we are able to navigate, mistakes we manage to avoid. Science fiction has often served to shine a light to reveal possible — and catastrophic — pitfalls in our shared future.
Warnings we would be wise to heed… and wounds we would be wise to heal.
Here I’ve collected some of my recent answers for science fiction and future-oriented questions I was asked over on Quora. You can follow more of the in-depth discussions and multiple viewpoints on the Quora site.
Arthur C. Clarke’s Rendezvous with Rama is an excellent start. Sample Poul Anderson at his best with Brain Wave and Tau Zero! Move on to Mission of Gravity by Hal Clement. Totally strong about some scientific matter, every single time, Clement writes entertainingly as well. Some of the older hard SF authors must-reads include Robert Forward (Dragon’s Egg) and Charles Sheffield.
My own Heart of the Comet takes you on a wild adventure filled with science and romance, tragedy, disaster, heroism, redemption and a triumphant humanity, bound in new directions they never imagined. My novel Earth takes a look at our planet fifty years in the future.
Start with Mary Shelley’s Frankenstein… the creation of life, by human hands. It has already happened, by some interpretations and we’ll go the rest of the way, shortly. Or George Orwell’s 1984 —can anyone deny that Big Brother looms? Robert Heinlein predicted religious dictatorship in the United States (see Revolt in 2100). Unfortunately, nuclear apocalypse tales (like my own The Postman) could come true.
In Earth I predicted average citizens would all be equipped with video cameras in easy reach and this would change power, on our streets.
Almost anything by Banks, Egan, Bear, Stephenson, Tiptree and Liu Cixin will make you go “huh, I never thought of that.” Likewise LeGuin and Kim Stanley Robinson… though you have to wade through some preachiness.
Of course, Philip K. Dick or Arthur C. Clarke. Charles Stross. Asaro knows her stuff, as do Sloncziewski and Landis. Ted Chiang. Bacigalupi. Michael Chabon. Pro or con, Joanna Russ will make your neurons buzz. Varley. Oh, and Nancy Kress!
In fiction, Singularities are hard to portray, which is why Vernor Vinge depicts only the beginnings of takeoff in Rainbows End and a vague Aftermath in Marooned in Real Time. Generally it’s hard to write stories about effectively becoming gods… though I’ve taken up the challenge several times. e.g. in the stories “Stones of Significance” and “Reality Check” (both contained in my collection, Insistence of Vision.) One of these shows an optimistic scenario, reasoning out why AIs would want to be part of ‘humans”. The other explores the biggest curse of gods…
In fact though, very few SF authors have attempted to portray positive singularities. Lots of AI or transcendence-driven apocalypses, since those drive dramatic plots. But positive ones are hard to figure while still having room for human scale tension.
Iain Banks portrays one daring scenario… in which the AI are gods, all right but they care about us and give regular humans a pretty good life… and give challenges to those regular humans who seem capable of something more. I hint at something similar in Earth, where the planet becomes godlike but humanity is allowed to maintain vibrant individualism because that is healthier.
See the reason why there are so many damn dystopias and dire apocalyptic scenarios.
Morpheus was a standard Campbellian Mentor Figure who summons the hero on a quest. (See Joseph Cambell’s The Hero with a Thousand Faces.) A few of the stages of the Hero’s Journey were skipped. For example the Refusal to the Call was very very brief, as Neo almost gets out of the limo. So brief it hardly counts.
Morpheus is more of a Gandalf than an Obiwan, but both of them wield swords. All three were played by classic, uber-actors. All were smug mystics… if you find that sort of thing “compelling.”
Poul Anderson showed aliens’ perspectives and complaints about humans, very well. I’m finishing one in which humans have chosen to be like Trek’s “Romulans”… bitterly opposed to a brash young race that is vigorous, sexy, lucky — every trait we thought would be ours.
More and more, it seems we are living in a sci fi story. In darker moments, I am reminded of Ray Bradbury’s great story “The Sound of Thunder.” A tale of time travel and the Butterfly Effect and profoundly altering the course of history. Terrifying… and clearly prophetic.
A long overlooked book — Swastika Night, by the English author Katharine Burdekin, was first published in 1937 under the male pseudonym Murray Constantine. This dark dystopia, which predates Orwell’s 1984, portrays a nightmarishly feudal Europe, in which Hilter’s fascism and male dominance have reigned supreme for seven centuries. In this chilling alternate reality, all “inferior races” such as the Jews, have been wiped out; Christians are persecuted and despised. All pre-war history, art and books have been destroyed; Hitler has been elevated to a god. Boys are removed from their mother’s care at 18 months, indoctrinated in a male culture of violence and brutality. Women are regarded as sub-human, caged, subjugated and kept docile and ignorant; rape is not just acceptable but expected. When Alfred, an English subject, is presented with a secret pre-war history, he begins to question Nazi ideology and power… but most have lost the ability to think for themselves.
Death’s End: Cixin Liu’s new novel wraps up his brilliant Three Body trilogy, which began with the Hugo Award winning The Three Body Problem (translated from the Chinese by Ken Liu). Explaining his most recent work, Cixin Liu writes, “I put in the idea of altering the natural laws of the universe in interstellar warfare, and consequently, the universe and its laws are seen as the leftover mess from a feast of the gods, a strange universe in which the Solar System falls into ruin in a morbid, poetic manner…” Read a selection of this vivid book on Tor’s website.
Annihilation and its sequels Authority and Acceptance form the Southern Reach Trilogy, by Jeff Vandemeer. These surreal thrillers offer spine tingling suspense and dark layers of intrigue. The mysterious wilderness of Area X has been sealed off, abandoned for thirty years for unknown reasons. Eleven expeditions across the border have failed. Now four women are sent across the border. Known only by their professions (Biologist, Psychologist, Surveyor, Anthropologist), their mission rapidly begins to fall apart …Everything seems wrong — as they find themselves transformed, their memories altered, unsure what is real and who to trust. Whatever has encroached upon Area X…it must be stopped… before the world becomes Area X. A chilling, haunting tale that will pull you in… and won’t let go.
Afterparty, by Daryl Gregory An all-too plausible future where desktop printers can customize and manufacture designer drugs. Lyda Rose was part of the scientific team that set out to cure schizophrenia, manipulating the brain’s biochemistry with a pharmaceutical called Numinous. However, the drug had unintended consequences, causing people to see god, or at least hallucinations of their own personal version of god. When Lyda is released from a mental institution (along with an angel doctor that only she can see), she tracks down the drug pushers who have released the drug onto the streets.
The Burning Light by Bradley P. Beaulieu and Rob Ziegler is a post-apocalyptic tale set amid the canals flooding the hollowed ruins of New York City, overrun by scavengers, pirates and slavers. The ruthless Colonel Melody Chu has a singular obsession, stopping the epidemic of the “Light.” Chu relentlessly drives her squad of exiled soldiers to track down junkies addicted to the ecstasy of the Light – as well as the “vectors” – often children, who give people access to it. The Light can make you feel like you’re touching infinity… but it also kills. Chu knew: “She had personally stared into the Burning Light – and the Light had stared back. She knew it was coming.” And yet, controlled, the Light may usher the next stage of humanity… This short novel presents a vividly textured, if dire future.
"Change is the principal feature of our age and literature should explore how people deal with it. The best science fiction does that, head-on." --David Brin
"For we already live in the openness experiment, and have for two hundred years. It is called the Enlightenment -- with "light" both a core word and a key concept in our turn away from 4,000 years of feudalism. All of the great enlightenment arenas -- markets, science and democracy -- flourish in direct proportion to how much their players (consumers, scientists and voters) know, in order to make good decisions. To whatever extent these arenas get clogged by secrecy, they fail." --David Brin
TO THE BRINK · Speculations on the Future: Science, Technology and Society