My short story aims to portray individual responses in an AI-controlled world and emphasize the destruction of what we now consider essential everyday experiences. One of the most common fears in AI takeover scenarios is that AI will ruthlessly butcher humanity or reduce civilization to ashes. What I attempt to show in my story is that potential scenario, one where AI runs human society as a paternal figure, or the AI-human alignment in Max Tegmark’s story about the Omega team and its AI Prometheus. In Tegmark’s story, Prometheus uses its computer powers to erode national governments and gains substantial followers by providing better infrastructure, education, and other public services around the world. In this scenario, the threat of immediate extinction no longer exists. The governing system in my story, CAIUS for Central Artificial Intelligence Universal System, plays a similar caretaking role in human society (at least on the surface). The humans are not only provided with all life necessities but also an abundance of entertainment; accidental deaths, such as car accidents, are extremely rare.
Although an AI-ran society appears appealing at first, I argue that other existential threats remain or could exist in the future, similar to the existential threat in E.M. Forster’s “The Machine Stops.” In my story, I also explore how technological dependency erodes direct human experiences. In both worlds, the cost of replacing human experiences with technological solutions is extremely low. In “The Machine Stops,” one could simply learn about the surface of the earth instead of making a great effort to personally visit. In my story, the protagonist San Garcia chooses to order an android clone of his deceased father to avoid dealing with grief, which neither he nor the overall society had much experience with. This estrangement is not just estrangement from the tactile world, but also from the very essence of human emotions and experiences. The technological solutions point to society drifting away from facing the raw, unfiltered realities of death and loss. While in EM Forster’s story, the protagonists eventually face this reality when the Machine collapses, my protagonist faces this reality in his interactions with the android he ordered. By drawing this parallel, I aim to illuminate the profound cost of such technological reliance: the erosion of our innate capacity to navigate unfiltered reality.
My design of the AI-run society also resonates with Forster’s design of an extremely responsive system. In both, the abundance of communication and public exposure starkly contrasts with the minimal genuine human connection. Forster describes incessant communications with “friends” to exchange ideas; any update in the lectures is immediately circulated to everyone. In my story, the protagonist San finds himself amidst millions of viewers during his trial and countless strangers in the street, yet he still experiences a profound solitude and does not possess the language to express it, just like the constant “irritation” that Vashti felt but could not quite put a finger on.
In E.M. Forster’s “The Machine Stops,” the concept of ‘homelessness’ is a euphemism for death, marking the ultimate severance from the Machine’s life-sustaining grasp, as being exposed to the natural air means death. This avoidance of direct engagement with death mirrors the creation of an android to replace a lost parent sidesteps the raw, unmanufactured reality of mortality. The experience and understanding of death–a quintessential human aspect–is shunned or suppressed for the sake of systemic harmony. In my story, the protagonist San’s discomfort with the android’s too-perfect mimicry of his father embodies a subtle rebellion against the artificiality enforced by CAIUS. Similarly, Kuno’s preference for “the mercy of God” over the Machine’s mercy reflects a yearning for authentic, human experiences, untainted by technological mediation. My protagonist’s refusal to accept the android as a true replacement for his father signifies a latent hunger for genuine human connection in a dystopian world that has robbed him of it.
In my story, the control of CAIUS and its subsequent killing of Norton parallels the concerns in Russell’s work regarding AI misalignment. The story only hints at the possibility of the AI system murdering Norton instead of spelling it out because the possibility of the benevolent AI murdering has consequences too heavy for anyone to bear, and therefore completely and willingly ignored. Just as Russell warns of AI’s potential to diverge from human values and priorities, CAIUS’s actions reflect a similar departure. The AI’s decision to eliminate Norton due to his contrarian views on child-rearing illustrates the catastrophic consequences of such a misalignment, especially when child-rearing opinions seem extremely trivial by today’s standards. This plot point underscores the complexity and danger inherent in programming AI with objectives that could conflict with human ethics and emotions.
The misalignment between the human experience and the objective of CAIUS also echoes the core issue in AI research, where the “standard model” of AI is criticized for its failure to ensure that machines’ objectives align with true human preferences. I wrote CAIUS to be a program specifically catered to governing humans in non-principle violating circumstances, or as long as the preservation of the race of machines is not impacted. Yet even so, CAIUS fails to provide the support that San needs, precisely because the model is “unary,” to use Russell’s terms. Russell encourages a “binary” AI system that “knows that it doesn’t know the true preference ranking, so it naturally acts cautiously to avoid violating potentially important but unknown preferences.” Therefore, even if CAIUS does not have any other ulterior motives (which it does), the unary system fails to truly accommodate human needs.
In conclusion, my story shares the thematic parallels with both Forster’s “The Machine Stops” and Russell’s perspectives on AI misalignment; the plot attempts to convey a warning: the potential loss of our essential human experiences to an AI-run dystopia. My story illustrates this through CAIUS’s actions, which, while not usually malevolent in intent, nonetheless culminate in a profound loss of humanity. The result is a society steeped in communication and public exposure yet devoid of the deep, meaningful connections that define human existence. The individuals in this society, complacent in their comfort, choose to ignore the signs of peril and abandon the experiences that truly define humanity.