“Our bondage will steal upon us noiselessly and by imperceptible approaches.”
The Latin word for “word” was originally used in Latin. genius was more readily applied to places — genius loci: “the spirit of a place” — than to persons, encoded with the reminder that we are profoundly shaped by the patch of spacetime into which the chance-accident of our birth has deposited us, our minds porous to the ideological atmosphere of our epoch. It is a humbling notion — an antidote to the vanity of seeing our ideas as the autonomous and unalloyed products of our own minds.
It has always been this way in all cultures, at all epochs. In ours, its most menacing manifestation — both unflattering and alarming — is something unprecedented: We are now porous not only to the collective ambience of human thought, but also to something half-human, something sub-human: It is neither you nor I deciding which of the photographs and poems I post on my Instagram you get to see; the algorithm that decides for you is not sentient in the sense that you and I are. It was once composed in code by human hands moved by human minds, and now it steers the bottom line of a human-governed company, but at that moment, that inflection point where it metes out your allotment of cultural material, it is pure machine — an automaton of variables, not one of them visible to you, not one controllable, together shaping what truth and beauty may appear before you, feeding what you may think about today and dream about tonight and dream up tomorrow or next year, furnishing the building blocks of your own genius.
On June 13, 1863, a letter was printed in a New Zealand newspaper under the heading “Darwin Among the Machines,” by someone who signed himself Cellarius and who later turned out to be the English writer Samuel Butler (December 4, 1835–June 18, 1902). At only twenty-seven, a century and a half ahead of his time, Butler prophesied the future of what we now call artificial intelligence and what he, epochs before the first modern computer and the golden age of algorithms, called “mechanical life” or “the mechanical kingdom.” Radiating from his visionary thought experiment is a calm, lucid admonition about what it would take to preserve our humanity — our singular human genius — amid this sea change changing the very fabric of consciousness.
Butler begins:
The amazing improvements in mechanical devices are something of which our generation can be proudest. It is a matter to be commended on many levels. They are obvious enough. But our current concern is with factors that might cause us to feel humbled and encourage us to think about the future. If we revert to the earliest primordial types of mechanical life, to the lever, the wedge, the inclined plane, the screw and the pulley, or (for analogy would lead us one step further) to that one primordial type from which all the mechanical kingdom has been developed, we mean to the lever itself… we find ourselves almost awestruck at the vast development of the mechanical world, at the gigantic strides with which it has advanced in comparison with the slow progress of the animal and vegetable kingdom. It will be impossible not to ask ourselves the question: What is its end? Which direction is this movement heading? Is it going to be successful?
Proceeding “to give a few imperfect hints towards a solution of these questions,” he adds:
A new kingdom was created in the last few years. As yet, we have not seen any of what would be called the antediluvian examples of the race.
A century before Gordon Moore drew on his work with semiconductors to formulate his eponymous law for the exponential shrinking and acceleration of technology over time, Butler observes the unprecedented pace at which this “kingdom” of near-life has emerged:
The size of some vertebrates that were lower than others has contributed to their growth and progression. Consider the watch as an example. Examine the beautiful structure of the little animal, watch the intelligent play of the minute members which compose it; yet this little creature is but a development of the cumbrous clocks of the thirteenth century — it is no deterioration from them. Watches may soon replace clocks. Although clocks aren’t shrinking in bulk at present, they may become obsolete in the future. In that case, clocks may disappear like earlier saurians. The watch, whose size has tended to decline over the years, will be the last extinct type.
One need only follow this progression to its logical conclusion to face the inevitable question of “what sort of creature man’s next successor in the supremacy of the earth is likely to be”:
Our own success is ours. We add beauty to their bodies and provide them with the power to act and self-regulate. We will be the second-lowest race over the coming centuries.
But Butler’s bravery defies the techno-utopians against techno-dystopians. Inside his cautionary vision pulsates a childlike optimism — this was, after all, the infancy of the machine age — that our machines might become not only superior in power but superior in moral might: capable of supreme self-control with cognition “in a state of perpetual calm,” afflicted with “no evil passions, no jealousy, no avarice, no impure desires,” free from the notions of sin and shame that so savage human behavior. The cost of this higher consciousness, however, would be our ceaseless servitude — we would have to maintain the machines, fix their every malfunction, and “feed” their unremitting appetites. (It is curious, haunting even, that Butler uses the word “feed” a century and a half before it became the standard term for the machine-selected cultural matter served to our consciousness by our social media, to become the very content of our thoughts, beliefs, and values.)
Butler discusses the consequences of such codependence.
The state of things that we are attempting to describe will be reached when man is able to domesticate his animal under the beneficent rule of the machines. He will continue to exist, nay even to improve, and will be probably better off in his state of domestication under the beneficent rule of the machines than he is in his present wild state… Our interests are inseparable from theirs, and theirs from ours. For innumerable other benefits, each race depends upon the other. And, up until our reproductive organs can be developed in a way which is difficult for us to comprehend, we will not be able or able to reproduce, so they depend entirely on the man for the survival of their species.
Butler closes with an uncompromising prescription for the only route to self-salvation — also childish, as all absolutism in the face of complexity is, but at the same time more mature than what our present self-infantilized civilization is capable of conceding:
However, machines are gradually gaining on us day by day; we become more dependent to them. More men are being made slaves for them and more people are dedicating their entire lives to developing mechanical life. The upshot is simply a question of time, but that the time will come when the machines will hold the real supremacy over the world and its inhabitants is what no person of a truly philosophic mind can for a moment question… war to the death should be instantly proclaimed against them. All machines of all types should be destroyed by the best wisheser of their species. Let there be no exceptions made, no quarter shown… If it be urged that this is impossible under the present condition of human affairs, this at once proves that the mischief is already done, that our servitude has commenced in good earnest, that we have raised a race of beings whom it is beyond our power to destroy and that we are not only enslaved but are absolutely acquiescent in our bondage.
Butler was afraid that human civilization would collapse in 1863. He spent the next nine-years analyzing this prediction and imagining alternative futures. Erewhonians were able to rescue themselves by enacting the bold proposition that Butler called “Darwin Among the Machines” and banned all mechanical devices.| public domain) — the story of a contemporary traveler who, by some unnamed accident of spacetime, finds himself a visitor to a strange kingdom in a remote corner of Earth, inhabited by a self-contained culture that had long ago reached more advanced stages of civilization than ours, but sensed this impending enslavement by technology in the recognition that “the machines were ultimately destined to supplant the race of man, and to become instinct with a vitality as different from, and superior to, that of animals, as animal to vegetable life”; Erewhonians had managed to save themselves — to save their moral spirit, their happiness, and the life of the mind — by enacting the radical proposition with which Butler ended “Darwin Among the Machines,” banning all mechanical devices whatsoever.
In the novel, Butler builds on the ideas laid out in his essay and, contrasting the slow evolution of life and consciousness on Earth with the rapid evolution of machines, composes what is essentially a stunning warning label for what we now call artificial intelligence — the next stage of consciousness:
The earth looked like a hot, round, molten ball. It then cooled to the point that it became utterly starved of all life. If a human had ever existed in the present state of the earth, was he allowed to view it like it was another world, and was he completely ignorant of physical science? Would he have said that it was impossible for creatures with consciousness to evolve from this cinder? Is it possible that the earth could contain consciousness? But consciousness did emerge over time. If consciousness is not evolving, it could be possible to find new pathways for our awareness. However, we are unable to detect any signs.
Again. Consciousness, in anything like the present acceptation of the term, having been once a new thing — a thing, as far as we can see, subsequent even to an individual centre of action and to a reproductive system (which we see existing in plants without apparent consciousness) — why may not there arise some new phase of mind which shall be as different from all present known phases, as the mind of animals is from that of vegetables?
Inasmuch it is impossible to describe such a mental condition (or whatever its name may be), it seems absurd. But, surely, when one considers the many phases of consciousness and life that have already evolved, we would not be foolish to conclude that there are no other ways to develop them, or that animal life represents the end all of things. Fire was once the final thing.
At the heart of Butler’s thought experiment is an invitation, repeated almost the way in meditation one is continually invited to return to the breath, to consider the alarming rapidity with which mechanical proto-consciousness has emerged and already begun dominating tasks that organic consciousness has spent eons evolving for. Our machines have already become conscious because of the interconnectedness between our tasks and theirs. “Where does consciousness begin, and where end?” he asks. “Who can draw the line?… Is not everything interwoven with everything?” With an eye to these disquieting questions, and to the grimly shortened arrow of evolutionary time, he writes:
Higher-order machines tend to have more of yesterday than the past five minutes. Consider that conscious beings are known to have been around for approximately twenty million years. Now, consider the strides made by machines in the last 1,000 years. Can the world survive twenty-million years? What will the world become if they do? Do you not think it is safer to reduce the damage and stop them from making further progress?
[…]
It must always be remembered that man’s body is what it is through having been moulded into its present shape by the chances and changes of many millions of years, but that his organisation never advanced with anything like the rapidity with which that of the machines is advancing.
Once again epochs of thought ahead of his time — a time when “God” was considered the creator of all life and life was thought to be of metaphysical rather than physical fundament — Butler alludes to Hermann von Helmholtz’s discovery, a decade earlier, of the speed of electricity across human nerve fibers, intimating that if the basic infrastructure of consciousness as we know it and feel it is but a matter of electricity across wires, then our mechanical companions are not so far removed from the concept of consciousness: If every sensation is “chemical and mechanical in its operation,” why do we think that “those things which we deem most purely spiritual are anything but disturbances of equilibrium in an infinite series of levers, beginning with those that are too small for microscopic detection, and going up to the human arm and the appliances which it makes use of?”
One of the novel’s characters captures the corollary of these questions in a sentiment that may well be — and perhaps must be — aimed at the fundamental assumptions of our own time:
None of these machines are in danger; I am afraid only that they will evolve at an extraordinary rate. In the past, no class of human beings has made such a rapid progress. This movement should be closely monitored and checked as soon as possible. It is necessary that we destroy advanced technology, which is a fact even though these machines are harmless.
[…]
We cannot calculate on any corresponding advance in man’s intellectual or physical powers which shall be a set-off against the far greater development which seems in store for the machines. Some people may say that man’s moral influence will suffice to rule them; but I cannot think it will ever be safe to repose much trust in the moral sense of any machine.
Butler writes:
We will be robbed of our bondage by noiseless and imperceptible methods.
Butler’s predictions seem to have proved correct a century later. The thought that we might be unable to control our humanity is chilling. This remains a very open question and must be addressed with our lives. Each act of resistance is important.
You can complement Nick Cave with music, feeling and transcendence in this age of AI. Next, visit H.G. Wells’s prophetic vision for the “World Brain.”
Giving = Being Loving
Over the past decade, I spent hundreds of hours writing and thousands of dollars each monthly. MarginalianThe magazine, which bore for fifteen years the unsettling name Brain Pickings. The site has survived despite being ad-free, and thanks to readers’ patronage it is still free. I have no staff, no interns, no assistant — a thoroughly one-woman labor of love that is also my life and my livelihood. Donations are a great way to make your life better. Every dollar counts.
Newsletter
MarginalianGet a weekly, free newsletter. It comes out on Sundays and offers the week’s most inspiring reading. Here’s what to expect. Like? Sign up.
