Killing ourselves with technology

What do we do when technology spirals out of our control? Or, to put it more bluntly, when does humanity’s ability to build ever more dangerous weapons become a self-fulfilling prophesy?

Albert Einstein is said to have remarked that he didn’t know what weapons the third world war would be fought with, but the fourth would be waged with sticks and rocks. Even that classic of science fiction optimism, Star Trek, had humanity surviving a third world war. (Spock recounted the tolls of Earth’s three world wars in one episode.)

But we wouldn’t, would we? Or we might wish we didn’t. One story that has long lingered in my mind is an early Philip K. Dick story, “Second Variety,” published in 1953, a time when the cold war was looking decidedly hot. The story takes place in a post-apocalyptic France, in a world in which nuclear bombs and other equally nightmarish weapons have reduced most of North America and Europe to gray ash, with only a stubby tree trunk or a blasted wall dotting barren, depopulated landscapes.

NagasakiThe West’s governments have retreated to a bunker somewhere on the Moon, with scattered groups of soldiers huddled in hidden underground bunkers on Earth trying to “win” the world war. The land is uninhabitable because of a super-weapon developed by the U.S. — autonomous machines that hone in on any living being and rip it to shreds with whirring metal blades that make short work of whatever they encounter. The Western soldiers are protected by a belt that forces the death machines to back off. This is the weapon that turns the tide of the war into a U.S. advantage after years of “losing” the war against the Soviet Union.

But what is there to “win”? Much of the world is uninhabitable, not only because of the total destruction and residual radiation from countless bombs but from the new weapon. There is no alternative but to huddle in underground bunkers. As Dick’s story unfolds, the nightmare gets progressively worse — the weapons are not only autonomous, they are self-replicating and continually inventing newer and more deadly varieties of themselves. The last pockets of U.S. and Soviet soldiers in this slice of the French countryside are systematically killed as the machines learn to build robots difficult to distinguish from humans; robots allowed into bunkers as refugees, only to suddenly become unstoppable killing machines, and which don’t distinguish one side from the other.

Although shuddering at the mere thought of their deadliness, more than once a soldier tries to justify these ultimate weapons by saying “If we hadn’t invented them, they would have.”

If we didn’t shoot first, bomb first, destroy first, they would have. Whatever we do is justified. No culture has a monopoly on such thoughts. But such thoughts combined with the technological progress of the present day, rising nationalism and budget-busting military budgets leave the possible end of the human race a concrete possibility rather than merely a science fiction allegory.

Philip D. Dick was no prophet — no one is — but the nightmare world he created is chillingly tangible. What would happen if a technology of war was given autonomy? Such a weapon would be purposefully designed to kill swiftly and without mercy. The Pentagon has already begun a program designed to create autonomous weapons systems.

(Cartoon by Carlos Latuff)

(Cartoon by Carlos Latuff)

But what if an artificial intelligence decided humans were in the way? Isaac Asimov famously had his robots programmed with three laws that blocked them from doing any harm to any human. The other side of this equation was explored in another Star Trek episode, when the Enterprise encountered a planet populated by advanced robots. The robots had killed their creators so far back in time that the robots couldn’t remember when, but had done so because their creators “had begun to fear us and started to turn us off.”

Technology need not be feared nor is it necessarily fated to escape all control. There are no von Neumann machines swarming everywhere (at least in this part of the galaxy!), and I am inclined to agree with Arthur C. Clarke’s maxim that there is no evil technology, only evil applications of technology. Yet we live in a world where there are plenty of opportunities for technology to be used for evil purposes. We see some of this all around us as workplaces become sites of tightening surveillance and control, from computers that report on us to bosses, to the endless treadmill of work speedups. Technology is today a tool of capitalists, to extract ever more work out of us, to outsource work on scales never before possible and to facilitate ever faster and more numerous speculation in dubious financial instruments.

Technology in these hands also makes waging war easier — a drone operator can sit in a control room thousands of miles from the targets, safe from the carnage rained down on far-away peoples. If autonomous weaponry ever is unleashed, how could it be controlled? It couldn’t. Humanity won’t survive a third world war.

When we think of existential threats to our descendants’ world, we tend to focus on global warming, environmental degradation and the looming collapse of capitalist industrialism, of the impossibility of infinite growth on a finite planet. That is properly so, and these do seem to be the gravest challenges that will face us across the 21st century. But technology applied to perfecting military killing machines is within the human imagination. Dick conjured this at the midpoint of the 20th century and he is far from the only one.

Yes, a warning and not a prophesy. But in a world of vast inequality, of an industrial and financial elite willing to do anything, even put the planet’s health at risk, for the sake of acquiring more wealth, the potential for evil applications of technology are ever present.

One more reason, if we didn’t already have enough, to bring into being a better world, one built for human need and environmental harmony rather than private profit. We then wouldn’t need to endure a mad pursuit of fetishized technological advancement; instead we could harness technology for the greater good as necessary. Barbarism remains the likely alternative.


39 comments on “Killing ourselves with technology

  1. newtonfinn says:

    “We may say, therefore, that modern technology has deprived man of the kind of work that he enjoys most, creative, useful work with hands and brains, and given him plenty of work of a fragmented kind, most of which he does not enjoy at all. It has multiplied the number of people who are exceedingly busy doing kinds of work which, if it is productive at all, is so only in an indirect or ’round-about’ way, and much of which would not be necessary at all if technology were rather less modern. Karl Marx appears to have foreseen much of this when he wrote: ‘They want production to be limited to useful things, but they forget that the production of too many useful things results in too many useless people,’ to which we might add: particularly when the processes of production are joyless and boring. All this confirms our suspicion that modern technology, the way it has developed, is developing, and promises further to develop, is showing an increasingly inhuman face, and that we might do well to take stock and reconsider our goals.” E.F. Schumacher, “Small is Beautiful,” published (presciently) in 1973.

  2. Steph free says:

    If only a human-only virus would come along, & leave the rest of the creation alone…

    • Cybersyn says:

      Then the rest of “creation” would destroy itself since all life (on Earth) has an inclination toward violence, war and self-destructive behavior,

      • Night-Gaunt49 says:

        Nature is about balance and hadn’t destroyed itself in over a billion years. Though there are other forces in Nature which could wipe out all or most life. Like the one we have precipitated by accident—AGW. A Type I level event from a Type 0 level civilization (us) can accomplish.

        Look up the other aspect of Evolution under “Mutual Aid”.

  3. Cybersyn says:

    In later iterations of Star Trek it’s said WW3 killed nearly a billion people and caused society for a long time to go into a Mad Max style hellhole, with what’s left of governments executing people in mass for radiation damage and mutations. In other words, when that’s one of the brighter looks at WW3, you know you’re fucked (if WW3 happens anyway).

    • It was “only” 80 million or something like that in the original Star Trek series, and no “Mad Max” degeneration. The billion-deaths figure sounds like it would have been said in the J.J. Abrams reboot, which substituted thought and character relationships with the standard Hollywood blow everything up, special effects fetish. Not to mention total discontinuity with the original series and the later sequels.

      If civilization were to collapse into a “Mad Max” scenario (which I do not think will happen), there would be no recovery. Barbarism in a world completely stripped of natural resources and/or destroyed in a third world war would be final. But, as the famous saying by Fredric Jameson says, it’s easier for people to imagine the end of the world than the end of capitalism. So true, unfortunately.

      • Cybersyn says:

        Don’t argue with a Trekkie (or Trekker? ..whatever) on Star Trek :P. Apparently it was 600 million: so I guess I was stretching it when I said almost a billion. But then again this is from the TNG onward iterations (which I far prefer over the original) and not the original, so obviously there’s massive contradictions. The Mad Max style anarchy comes from descriptions in TNG and Enterprise according to Memory Alpha. Actually it sounds a lot worse than Mad Max, which reduces some post apocalyptic Australia into a giant fun action zone.

        No argument on your description of the Abrams Trek films, but as far as I know, they’ve never even brought up WW3 or any actual cerebral or sci-fi concepts ever, let alone concepts rom the other five series and other movies, unless it’s to take and make into some Hollywood bullshit (Khan for example). They’re honestly the only iterations of Trek I truly dislike, and I can stomach some of Voyager and Enterprise, so they’re really bad…

        If you think the world is doomed, then I think your blog and writings and “mission” are kind of self-defeating. I wouldn’t waste one moment of my time trying to improve anything if I thought it was just going to burn to the ground. Then again I’m highly skeptical of end of the world scenarios, including (and especially) ones linked to climate change, so I’ve never had this thought.

      • Cybersyn says:

        One last thing on all this pointless discussions of fiction: I think Mad Max is a terrible franchise (the latest movie was ok admittedly) because not only are they just bad at action and totally silly, it’s stuff like that that presents unrealistic depictions of “collapse” and feeds into cynicism and pessimism that annoys the shit out of me. And Australian accents are annoying.

        As for recovery, well in Star Trek the only reason why Earth ever recovers is the Vulcans, which is actually a pretty frightening message, that our only hope is more enlightened beings that will give us a helping hand. I could only hope that would happen after total societal breakdown and nuclear radiation.

        • I hadn’t realized that World War III was discussed so much in the series following the original. I found The Next Generation boring and rarely watched any of the others.

          In reading through your link, I see that Vulcans lending a helping hand were what enabled Earth to recover from 27 years(!) of nuclear war. It’s hard to imagine there’d be anything to recover after that. And the thought that only a lucky break from a technologically superior planet could save the day is the sort of idea that encourages inertia. It is a frightening message, and depressing.

          Just for the record, I am also skeptical of end of the world scenarios, and I disagree with peak oil people that there will soon be a dramatic crash. What I do believe is that there will be a disintegration across decades (a period marked by stagnation with downward lurches) into a world overwhelmed by rising oceans, heavy pollution, resource depletion and massive displacement unless there is a mass social movement on a global basis that intervenes. That is not the only scenario if capitalism is left to act on its own momentum, but I do believe it is a probable scenario.

          It is precisely because I do not believe that disintegration scenario is a certainty — instead, that we can intervene and bring into being a better, more rational, stable and sustainable world — that I write this blog and regularly engage in other ways, including activist work. If I believed collapse was inevitable, I’d save myself a lot of work by doing nothing.

          • newtonfinn says:

            While disintegration of global capitalism is not a certainty, Brexit certainly provided some evidence of decay.

            Nevertheless, I have become increasingly concerned that many on the left have come to identify more with lifestyle issues than with economic issues, not to denigrate in any way the ongoing need for new forms of personal liberation. Numerous leaders in the British left, for example, were unable to even get near the “leprous” right wing (and, yes, bigots, among others, are there) for the sake of winning one absolutely crucial vote, which put a significant stake in the heart of the global capitalist vampire embodied in the EU.

            Despite the humiliation and immiseration of our brothers and sisters in Greece, gleefully celebrated by their EU overlords, the “purity” and “self-righteousness” of these leftist leaders–their public and private image–proved more important to them than a temporary alliance with a national enemy to defeat an international monster, one that has been feeding on 99% of the British public, not to mention the entire planet. Small wonder that the momentum for change is sadly shifting to the right. When will the left recover its passion and its guts?

            • newtonfinn says:

              Replying to myself because no one seems to want to talk about perhaps the most telling blow yet struck against global capitalism, let me reiterate that technocracy (the subject, in part, of the lead essay) and capitalism are blood brothers…of the sucking kind. Here’s the best Brexit analysis I’ve come across concerning left wing timidity. And it was written before the vote.


            • I believe the Dissent author is overstating Left support for Britain remaining in the EU. And as I am not British, I don’t see it as appropriate for me to tell them how to vote. Certainly the point that there is a Left argument to be made for EU withdrawal is true, and that can be done while disassociating from the Right-wing xenophobia depressingly on display.

              A better argument would be for all countries to withdraw from the EU; one particular country isn’t going to make much difference, if any (except for Britain itself). What is likely to happen is that Britain will find itself with much less “independence” from Europe that Brexit advocates suppose — to avoid disruption to the economy, it is probable that Britain will negotiate an agreement similar to those the EU has with Switzerland and Norway, both of which adhere to most EU regulations and have to pay into EU programs to have regular trade.

              A single country’s withdrawal from the EU is a reform, nothing more, and much less of a reform than Brexit advocates reckon. Smashing the anti-democratic, neoliberal EU is a continent-wide project, not one for any single country.

              • newtonfinn says:

                Perhaps you are right, perhaps not. Let’s see (and hope) that the dominoes do indeed drop. Distancing oneself from the life and death issues of the struggling working class, as many on the left have done, because some of its members have regressed (often through manipulation) to bigotry and xenophobia, reminds me of the old line that liberals love the poor but not the smell of them. No one put his finger on this issue better than the late, great Joe Bageant.


          • Cybersyn says:

            Ironically, even though in ultra nerd circles I’d get lynched for this, I find the Original the most boring and unwatchable (though I don’t hate it by any means) since it’s so old and cheesy and out of date (and the only Trek that has bizarre, cringey jingoistic and pro war overtones…yeah), even compared to the more “hated” later series that are supposed to suck major ass according to popular opinion, yet imo aren’t so bad. But anyway, enough of this, this isnt Reddit or TrekBBS or YouTube. As for the recovery, it actually takes 100 or so years in the Trek canon, it”s just unseen.

            Well, I’m skeptical of even nuclear war automatically leading to such destructive scenarios, Freeman Dyson for example has written on this. Not that I’m in favor of nuclear war obviously, but I don’t think it’d automatically lead to Fallout or the extinction of the human species, or complete societal destruction, so ironically I don’t actually think we’d need a super enlightened friendly race to recover from it. All in all, I’m highly skeptical of nuclear war end time scenarios, even though I’m more afraid of nuclear war than say climate change.

            Fair enough if you don’t think it’s inevitable, I’m just glad I’m not consumed by such thoughts and ideas, since I don’t find them very valid.

            • Cybersyn says:

              I think this is why I’m not a “radical” in the true sense of the word, and even my socialism is , while “anti-capitalist”, not that radical or extreme (I am a Democratic Socialist after all, and I was calling myself that long before this election cycle), I realize the extreme ends are just nonsense for the most part and I don’t invest in them. I guess it’s also why I have zero interest in activism and don’t want to get heavily involved in politics, and would rather just comment and rant about it on the sidelines. Politics and activism are a lot more boring when you realize the world isn’t going to burn all around you, and the world doesn’t need you.

              • Cybersyn says:

                One very last thing I want to clarify before I drop this for good (and since I can’t edit comments, I’m stuck with multiple posting, sorry): the only reason that particular aspect of Star Trek is scary is that in reality there is no white knight coming to save us and bail us out with an intergalactic Marshal Plan, that’s what’s frightening, I wasn’t making any comment on the human condition. Other than that it’s one of my favorite aspects of the franchise, since it’s about cooperation and friendship and not Independence Day style invasions and blowing shit up. I wish there were friendly aliens coming to Earth helping us, but alas, there isn’t.

                Anyway, this whole election cycle and year of exposing myself to radical leftist politics has given me an overload and I’m kind of sick of politics for the moment, I want to wind down and go back to what I usually immerse myself in: video games, snarky youtube videos, and bitching about the latest shitty movies, So I’ll stop spamming every other comment thread of yours from now on. Ciao.

          • Night-Gaunt49 says:

            Peak oil use is the greater danger. That and Human Made Global Warming getting worse as time goes on and we just sit on our thumbs while it does. Something never touched upon in the Star Trek universe. STNG got better as time went on. Still is wasn’t exuberant as the first iteration was. More sedate.

            • Cybersyn says:

              Peak Oil is not a danger much at all, as we’re not going through it. I’m more afraid of peak helium than I am peak oil, which is to say not at all.

              Actually TNG had a really bad analogy toward fossil fuel use and climate change in its last season, for a nerdy aside.

  4. Asteroid Miner says:

    Between 1945 and 1970, 600 nuclear bombs were exploded in the atmosphere of Earth. Nobody noticed. Why didn’t you die then?
    Because nuclear war is not what you think it is. You innumerate humanitologists are putting down engineers and scientists, probably because you flunked out of engineering.

    • kyoto2 says:

      Quite a lot of people did die – many of them unborn children, many more ordinary deaths from cancer that stand out from the statistical background – but only if you care to look.

      • Thanks for inserting some sanity into this conversation, Kyoto2. Like you, most of the world has noticed the explosion of nuclear bombs and the waste of nuclear energy. I’m glad some of us are willing to observe what we see.

        • Cybersyn says:

          I think nuclear weapons do have uses beyond killing people, so I actually wouldn’t get rid of them, I’d just get rid of their use as weapons.

  5. In a truly self-governing society there would be more citizen input – especially from women and minorities – into the unintended ecological consequences of new technology. At the moment most of the major technology advances are made by white men – and as Ronald Wright describes in A Short History of Progress, most have had devastating consequences for the planet.

    • Moti Nissani says:

      I doubt that will cure anything Stuart. You have for starters the British queen, Thatcher, Merkel. It seems to me that every race and gender has its fair share of psychopaths. The solution will have to be more radical and systemic, I think–see my comment below.

  6. Moti Nissani says:

    Excellent, thought-provoking, article. I’d go two steps farther than Peter: Until we establish real democracy (not the joke we have now–see link below), technology poses the greatest threat to freedom and human survival. And: In this world, it is immoral to be a scientist, or at least to make your results public. Speaking of Einstein; he reportedly said that, if he had to do it over again, he would choose to be a plumber.

    • Greetings, Moti, and thank you for the kind feedback. In his defense, Einstein isn’t responsible for the weapons that resulted from his discovery; he was a scientist building on those who came before him in the process of answering fundamental questions about the physical world.

      Although there are certainly moral lines that should not be crossed, scientific discovery in itself isn’t “immoral.” That is too broad of a brush.

      • newtonfinn says:

        I think Einstein died with great regret about what he had unwittingly helped bring to the world–the constant threat of nuclear war and planetary destruction. Draw your own conclusions from the last communication we have from him.

        “When Einstein died on April 18, 1955 he left a piece of writing ending in an unfinished sentence. These were his last words:

        In essence, the conflict that exists today is no more than an old-style struggle for power, once again presented to mankind in semi religious trappings. The difference is that, this time, the development of atomic power has imbued the struggle with a ghostly character; for both parties know and admit that, should the quarrel deteriorate into actual war, mankind is doomed. Despite this knowledge, statesmen in responsible positions on both sides continue to employ the well-known technique of seeking to intimidate and demoralize the opponent by marshalling superior military strength. They do so even though such a policy entails the risk of war and doom. Not one statesman in a position of responsibility has dared to pursue the only course that holds out any promise of peace, the course of supranational security, since for a statesman to follow such a course would be tantamount to political suicide. Political passions, once they have been fanned into flame, exact their victims… …………

      • Moti Nissani says:

        Thanks. You are right, science in essence is neutral. But, in the real world, every scientist must know (as did Leonardo when he invented the submarine and kept that invention to himself) that what he’s doing will be, almost certainly, turned against us. Computers? More surveillance and weapons. Organ transplants? Organ trafficking. Space program? Weapons, surveillance, loss of privacy. Nuclear power? Fukushima. Firearms? Genocides. Agriculture? Hierarchical society, shorter and hungrier people, slavery. Genetic engineering? Poisoned food, dangerous experiments (see, e.g., Taleb), and playing God. Cars? Climate disruptions, air pollution, health problems.

        Even the good inventions, in the present world, are turned against us. Tainted vaccines? Autism. Antibiotics? Poisoned animals and rivers. As long as a criminal syndicate is in charge of the planet, scientific work is itself immoral.

        Yes, now and then we have a discovery whose pluses overweight its minuses, but that is the exception.

        Think for instance: What would you say about a scientist that dedicated her life to, to her, the fascinating question of increasing the yield to weight ratio of nuclear bomb? You’d probably agree that this is immoral. Well, the same applies to all the natural sciences, and even psychology.

        As an ex-scientist I know and how harsh and Luddite this sounds. But I have no choice except judging science by its fruits. By this criterion, the ancient Greeks were right in being curious, but in not being particularly concerned about practical inventions.

        I am just about certain that the ancient Athenians (sadly, only citizens and men) were happier than we are. Anthropologists tell us time and again that Eskimos, or pigmies, without our dangerous science, where happier than we are.

        In the unlikely event that humanity survives its present predicament, our descendants might wisely banned all science, at least until they are able to create a rational system of governing themselves.

        Anyway, thanks for posting my comments, and I’d be looking forward to your next post!

        • Much to mull over here. On one level, I am sorry that you are an “ex-scientist” as we could use more scientists with a strong ethical compass. But no intention to second-guess the direction your personal development has taken.

          You write: “As long as a criminal syndicate is in charge of the planet, scientific work is itself immoral.” One the one hand, I’m rather glad we have things like electricity, which we wouldn’t have without science. On the other hand, the world is run in the manner it is and by the capitalist elite we have, and I have no argument with your assessment of them as a criminal syndicate. As long as the world is organized into hierarchies, with an elite suppressing the large majority of humanity to expand their wealth and power, science and technology will be used for bad purposes.

          I’d like to think it is possible to have a far different, better world, one based on equality, mutual respect and where everybody has enough. Science in that world — one where the mad pursuit of “innovation” as we experience it today is not necessary and is a part of the past — would be much more benign.

          • Moti Nissani says:

            Thanks. I now see your point more clearly, and largely agree with it.

            Here is a Facebook link I have just run across. I don’t know if what it describes is true. But if so, it shows that nations like the Iroquois did not only achieve greater social harmony, less cruelty, more freedom, happiness, sustainability, but even greater tolerance. All this—without science. Sure, I like computers and artificial light, but if reincarnation were an option, I’d choose to be born in that confederacy or, perhaps, Ancient Athens circa 380 or 410 (as a citizen). Here is the link, which might interest you:


            • I had run across this article myself. It begins:

              “It wasn’t until Europeans took over North America that natives adopted the ideas of gender roles. For Native Americans, there was no set of rules that men and women had to abide by in order to be considered a “normal” member of their tribe. In fact, people who had both female and male characteristics were viewed as gifted by nature, and therefore, able to see both sides of everything.”

              An enlightened outlook. During the 18th century, leaders of European colonies in North American lamented that many of their people ran away to join Native Americans, but the reverse never happened.

  7. MatthewHoh says:

    I’ve always liked the computers in Joe Haldeman’s “The Forever War”; after realizing that humans can’t be trusted, the computers take control and send mankind off through space on vacation…

  8. Joey says:

    I enjoy your writing, but must point out that the claim that “there is no evil technology only evil applications” is a fallacy – no technology is neutral, they are cultural artifacts, each being a reflection of the values of the culture and reinforcing the mindset of a culture. Modern technological gadgets reinforce the capitalist values of speed, efficiency, alienation, control and domination. Only a capitalist society, with the ideology of Western Science (“the world is dead” and humans are superior) to support it, could produce such inventions. Furthermore, no modern technology is possible without poisoning and destroying the Earth (mining, synthetic chemicals, factories, etc.), as well as destroying human communities to force them to do the mining and factory work that none of us technology-dependent Westerners would ever dream of doing. Regardless of use, technology is destroying us. It’s a reality few techno-dependent Westerners can face.

    • Yes, I agree. As I said in my penultimate paragraph:

      “[I]n a world of vast inequality, of an industrial and financial elite willing to do anything, even put the planet’s health at risk, for the sake of acquiring more wealth, the potential for evil applications of technology are ever present.”

      The environmental degradation from electronic gizmos, just from phones alone, is huge. We will have to do with less in the future, and indeed few in the advanced capitalist countries are ready to face that.

      • Cybersyn says:

        Doing with less is basically what austerity is all about, which is a pretty nightmareish future. If this is the case, I say bring WW3 or an asteroid on and put us out of our misery already. Then again I’m not an environmentalist so I’d disagree in any case. Maybe I shouldn’t have written off Peter Frase and Leigh Phillips, their techno socialism is probably more up my ally.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s