'The goal is to
automate us': welcome to the age of surveillance capitalism
Surveillance:
Vigilance,
oversight
Shoshana Zuboff’s new book is a chilling exposé of the
business model that underpins the digital
world. Observer tech columnist John Naughton explains the importance of
Zuboff’s work and asks the author 10 key questions
Chilling:
Frighting
Exposé: Report
Underpin: Support
John Naughton
Sun 20 Jan 2019 07.00 GMT
We’re living through the most profound
transformation in our information environment since Johannes Gutenberg’s
invention of printing in circa 1439. And the problem with living through a
revolution is that it’s impossible to take the long view of what’s happening. Hindsight is the only exact science in this business, and in
that long run we’re all dead. Printing shaped and transformed societies over
the next four centuries, but nobody in Mainz (Gutenberg’s home town) in, say,
1495 could have known that his technology would (among other things): fuel the
Reformation and undermine the authority of the mighty Catholic church; enable
the rise of what we now recognise as modern science; create unheard-of professions and industries; change the shape of
our brains; and even recalibrate our conceptions of childhood. And yet printing
did all this and more.
Hindsight: Understanding of a situation or
event only after it has happened or developed. (Mirada retrospectiva)
Unheard: Not heard previously
Why choose 1495? Because we’re about the same
distance into our revolution, the one kicked off by digital technology and networking. And although it’s
now gradually dawning on us that this really is a big
deal and that epochal social and economic changes are under way, we’re as
clueless about where it’s heading and what’s driving it as the citizens of
Mainz were in 1495.
Dawn on: Become evident to the mind. The
most habitual sense is the early morning: “From dusk till dawn”
That’s not for want of trying, mind. Library shelves groan under
the weight of books about what digital technology is doing to us and our world.
Lots of scholars are thinking, researching and writing about this stuff. But
they’re like the blind men trying to describe the elephant in the old fable:
everyone has only a partial view, and nobody has the whole picture. So our
contemporary state of awareness is – as Manuel Castells, the great scholar of
cyberspace once put it – one of “informed bewilderment”.
That’s
not for want of trying: Used for emphasizing that although someone did not get what they
wanted, they tried very hard to get it
Bewilderment:
Bemusement,
puzzlement
Which is why the arrival of Shoshana Zuboff’s
new book is such a big event. Many years ago – in 1988, to be precise – as one
of the first female professors at Harvard Business School to hold an endowed chair she published a landmark book,
The Age of the Smart Machine: The Future of Work and Power, which changed the
way we thought about the impact of computerisation on organisations and on
work. It provided the most insightful account up to that time of how digital
technology was changing the work of both managers and workers. And then Zuboff
appeared to go quiet, though she was clearly incubating something bigger. The
first hint of what was to come was a pair of
startling essays – one in an academic journal in 2015, the other in a German
newspaper in 2016. What these revealed was that she had come up with a new lens
through which to view what Google, Facebook et al were doing – nothing less
than spawning a new variant of capitalism. Those essays promised a more
comprehensive expansion of this Big Idea.
Endowed
chair: It would
be to reach this position
Landmark
book: Turning
point, critical point
Hint: Signal
And now it has arrived – the most ambitious
attempt yet to paint the bigger picture and to explain how the effects of
digitisation that we are now experiencing as individuals and citizens have come
about.
The headline story is that it’s not so much
about the nature of digital technology as about a new mutant form of capitalism
that has found a way to use tech for its purposes. The name Zuboff has given to
the new variant is “surveillance capitalism”. It works by providing free
services that billions of people cheerfully use, enabling the providers of
those services to monitor the behaviour of those users in astonishing detail –
often without their explicit consent.
“Surveillance capitalism,” she writes, “unilaterally
claims human experience as free raw material for
translation into behavioural data. Although some of these data are applied to
service improvement, the rest are declared as a proprietary behavioural
surplus, fed into advanced manufacturing processes known as ‘machine
intelligence’, and fabricated into prediction products that anticipate what you
will do now, soon, and later. Finally, these prediction products are traded in
a new kind of marketplace that I call behavioural futures markets. Surveillance
capitalists have grown immensely wealthy from these trading operations, for
many companies are willing to lay bets on our future behaviour.”
Raw
material: Material
still in its natural or original state, before processing or manufacture (matèria
primera, materia prima)
While the general modus operandi of Google,
Facebook et al has been known and understood (at least by some people) for a
while, what has been missing – and what Zuboff provides – is the insight and scholarship to situate them
in a wider context. She points
out that while most of us think that we are dealing merely with
algorithmic inscrutability, in fact what confronts us is the latest phase in
capitalism’s long evolution – from the making of products, to mass production,
to managerial capitalism, to services, to financial capitalism, and now to the
exploitation of behavioural predictions covertly derived from
the surveillance of users. In that sense, her vast (660-page) book is a
continuation of a tradition that includes Adam Smith, Max Weber, Karl Polanyi
and – dare I say it –
Karl Marx.
Covertly: Hidden
Viewed from this perspective, the behaviour of
the digital giants looks rather different from the roseate
hallucinations of Wired magazine. What one sees instead is a colonising
ruthlessness of which John D Rockefeller would have been proud. First of all
there was the arrogant appropriation of users’ behavioural data – viewed as a
free resource, there for the taking. Then the use of patented methods to
extract or infer data even when users had explicitly denied permission,
followed by the use of technologies that were opaque by design and fostered user ignorance.
Roseate: Pink
Foster: Promote
And, of course, there is also the fact that the
entire project was conducted in what was effectively lawless – or at any rate
law-free – territory. Thus Google decided that it would digitise and store
every book ever printed, regardless of copyright issues. Or that it would
photograph every street and house on the planet without asking anyone’s
permission. Facebook launched its infamous “beacons”, which reported a user’s online
activities and published them to others’ news feeds without the knowledge of
the user. And so on, in accordance with the disrupter’s mantra that “it is easier to ask
for forgiveness than for permission”.
When the security expert Bruce Schneier wrote
that “surveillance is the business model of the internet” he was really only hinting at the reality that Zuboff has now illuminated. The
combination of state surveillance and its capitalist counterpart
means that digital
technology is separating the citizens in all societies into two groups: the
watchers (invisible, unknown and unaccountable) and the watched. This has
profound consequences for democracy because asymmetry of knowledge translates
into asymmetries of power. But whereas most democratic societies have at least
some degree of oversight of state surveillance, we currently have almost no
regulatory oversight of its privatised counterpart. This is intolerable.
Counterpart: Someone's or something's counterpart is another person
or thing that has a similar function or position in a different place.
And it won’t be easy to fix because it requires
us to tackle the essence of the problem – the
logic of accumulation implicit in surveillance capitalism. That means that
self-regulation is a nonstarter. “Demanding privacy from surveillance
capitalists,” says Zuboff, “or lobbying for an end to commercial surveillance
on the internet is like asking old Henry Ford to make each Model T by hand.
It’s like asking a giraffe to shorten its neck, or a cow to give up chewing.
These demands are existential threats that violate the basic mechanisms of the
entity’s survival.”
Tackle: If you tackle a difficult problem or task, you deal
with it in a very determined or efficient way.
The Age of Surveillance Capital is a striking
and illuminating book. A fellow reader remarked to me that it reminded him of
Thomas Piketty’s magnum opus, Capital in the Twenty-First Century, in that it
opens one’s eyes to things we
ought to have noticed, but hadn’t. And if we fail to tame the new capitalist mutant rampaging
through our societies then we will only have ourselves to blame, for we can no
longer plead ignorance.
Tame: If you tame someone or
something that is dangerous, uncontrolled, or likely to cause trouble, you
bring them under control.
Rampage: When people or animals rampage through a place, they
rush about there in a wild or violent way, causing damage or destruction.
Plead: Allege
Ten questions for
Shoshana Zuboff: ‘Larry Page (*) saw that human experience could be Google’s
virgin wood’
(*) Cofounder of Google
John Naughton: At the
moment, the world is obsessed with Facebook. But as you tell it, Google was the
prime mover.
Shoshana Zuboff: Surveillance capitalism is a human
creation. It lives in history, not in technological inevitability. It was
pioneered and elaborated through trial and error at Google in much the same way
that the Ford Motor Company discovered the new economics of mass production or
General Motors discovered the logic of managerial capitalism.
Surveillance capitalism was invented around
2001 as the solution to financial emergency in the teeth of
the dotcom bust when
the fledgling company faced the loss of investor
confidence. As investor pressure mounted, Google’s
leaders abandoned their declared antipathy toward advertising. Instead they
decided to boost ad revenue
by using their exclusive access to user data logs (once known as “data
exhaust”) in combination with their already substantial analytical capabilities
and computational power, to generate predictions of user click-through rates,
taken as a signal of an ad’s relevance.
In the
teeth of: If you
do something in the teeth of a difficulty or danger, you do it in spite of the
difficulty or danger.
Fledgling:
You use
fledgling to describe a person, organization, or system that is new or without
experience.
Mount: Increase.
Ad revenue: Advertising incomes
Operationally this meant that Google would both
repurpose its growing cache of behavioural data, now put to work as a
behavioural data surplus, and develop methods to aggressively
seek new sources of this surplus.
Surplus: If there is a surplus of
something, there is more than is needed. Excess, extra.
The company developed new methods of secret
surplus capture that could uncover data that users intentionally opted to keep
private, as well as to infer extensive personal information that users did not
or would not provide. And this surplus would then be analysed for hidden
meanings that could predict click-through behaviour. The surplus data became
the basis for new predictions markets called targeted advertising.
Here was the origin of surveillance capitalism
in an unprecedented and lucrative brew: behavioural
surplus, data science, material infrastructure, computational power,
algorithmic systems, and automated platforms. As click-through
rates skyrocketed, advertising quickly became as
important as search. Eventually it became the cornerstone
of a new kind of commerce that depended upon online surveillance at scale.
Brew: A brew of several things is a
mixture of those things.
Click-through:
Do clic
with the mouse
Skyrocket: To rise rapidly
Cornerstone: The cornerstone of something is
the basic part of it on which its existence, success, or truth depends
The success of these new mechanisms only became
visible when Google went public in 2004. That’s when it finally revealed that
between 2001 and its 2004 IPO, revenues
increased by 3,590%.
IPO: Initial Public Ofference
JN: So surveillance
capitalism started with advertising, but then became more general?
SZ: Surveillance capitalism is no more
limited to advertising than mass production was limited to the fabrication of
the Ford Model T. It quickly became the default model for capital accumulation
in Silicon Valley, embraced by nearly every startup and app. And it was a
Google executive – Sheryl Sandberg – who played the role of Typhoid Mary, bringing surveillance capitalism from Google
to Facebook, when she signed on as Mark Zuckerberg’s number two in 2008. By now
it’s no longer restricted to individual companies or even to the internet
sector. It has spread across a wide range of products, services, and economic
sectors, including insurance, retail, healthcare, finance, entertainment,
education, transportation, and more, birthing whole new ecosystems of
suppliers, producers, customers, market-makers, and market players. Nearly
every product or service that begins with the word “smart” or “personalised”,
every internet-enabled device, every “digital assistant”, is simply a
supply-chain interface for the unobstructed flow of behavioural data on its way
to predicting our futures in a surveillance economy.
Typhoid
Mary: A
colloquial term for anyone who, knowingly or not, spreads disease or some other
undesirable thing. Mary Mallon (September 23, 1869 – November 11, 1938), also
known as Typhoid Mary, was an Irish-American cook. She was the first person in
the United States identified as an asymptomatic carrier of the pathogen
associated with typhoid fever. She was presumed to have infected 51 people,
three of whom died, over the course of her career as a cook. She was twice
forcibly isolated by public health authorities and died after a total of nearly
three decades in isolation.
JN: In this story of
conquest and appropriation, the term “digital natives” takes on a new meaning…
SZ: Yes, “digital natives” is a
tragically ironic phrase. I am fascinated by the structure of colonial
conquest, especially the first Spaniards who stumbled into
the Caribbean islands. Historians call it the “conquest pattern”, which unfolds in three phases:
legalistic measures to provide the invasion with a gloss
of justification, a declaration of territorial claims, and the founding of a town to legitimate the
declaration. Back then Columbus simply declared the islands as the territory of
the Spanish monarchy and the pope.
Stumble:
If you
stumble, you put your foot down awkwardly while you are walking or running and
nearly fall over.
Gloss: Attractive appearance, make-up.
The sailors could not have imagined that they
were writing the first draft of a pattern that would echo across space and time
to a digital 21st century. The first surveillance capitalists also conquered by
declaration. They simply declared our private experience to be theirs for the
taking, for translation into data for their private ownership and their
proprietary knowledge. They relied on misdirection
and rhetorical camouflage, with secret declarations that we could neither
understand nor contest.
Rely on: Trust
Google began by unilaterally declaring that the
world wide web was its to take for its search engine. Surveillance capitalism
originated in a second declaration that claimed our private experience for its
revenues that flow from telling and selling our fortunes to other businesses.
In both cases, it took without asking. Page [Larry, Google co-founder] foresaw
that surplus operations would move beyond the online milieu to the real world,
where data on human experience would be free for the taking. As it turns out
his vision perfectly reflected the history of capitalism, marked by taking
things that live outside the market sphere and declaring their new life as
market commodities.
We were caught off guard by surveillance
capitalism because there was no way that we could have imagined its action, any
more than the early peoples of the Caribbean could have foreseen the rivers of
blood that would flow from their hospitality toward the sailors who appeared
out of thin air waving the banner of the Spanish monarchs. Like the Caribbean
people, we faced something truly unprecedented.
Once we searched Google, but now Google
searches us. Once we thought of digital services as free, but now surveillance
capitalists think of us as free.
JN: Then there’s the
“inevitability” narrative – technological determinism on steroids.
SZ: In my early fieldwork in the
computerising offices and factories of the late 1970s and 80s, I discovered the
duality of information technology: its capacity to automate but also to
“informate”, which I use to mean to translate things, processes, behaviours,
and so forth into information. This duality set information technology apart
from earlier generations of technology: information technology produces new
knowledge territories by virtue of its informating capability, always turning
the world into information. The result is that these new knowledge territories
become the subject of political conflict. The first conflict is over the
distribution of knowledge: “Who knows?” The second is about authority: “Who
decides who knows?” The third is about power: “Who decides who decides who
knows?”
Now the same dilemmas of knowledge, authority
and power have surged over the walls of our offices, shops and factories to
flood each one of us… and our societies. Surveillance capitalists were the
first movers in this new world. They declared their right to know, to decide
who knows, and to decide who decides. In this way they have come to dominate
what I call “the division of learning in society”, which is now the central
organising principle of the 21st-century social order, just as the division of
labour was the key organising principle of society in the industrial age.
JN: So the big story
is not really the technology per se but the fact that it has spawned a new
variant of capitalism that is enabled by the technology?
SZ: Larry Page grasped
that human experience could be Google’s virgin wood, that it could be extracted
at no extra cost online and at very low cost out in the real world. For today’s
owners of surveillance capital the experiential realities of bodies, thoughts
and feelings are as virgin and blameless as nature’s once-plentiful meadows,
rivers, oceans and forests before they fell to the market dynamic. We have no
formal control over these processes because we are not essential to the new
market action. Instead we are exiles from our own behaviour, denied access to
or control over knowledge derived from its dispossession by others for others.
Knowledge, authority and power rest with surveillance capital, for which we are
merely “human natural resources”. We are the native peoples now whose claims to
self-determination have vanished from the maps of our own experience.
Grasp: If you grasp something that is
complicated or difficult to understand, you understand it.
While it is impossible to imagine surveillance
capitalism without the digital, it is easy to imagine the digital without
surveillance capitalism. The point cannot be emphasised enough: surveillance
capitalism is not technology. Digital technologies can take many
forms and have many effects, depending upon the social and economic logics that
bring them to life. Surveillance capitalism relies on algorithms and sensors,
machine intelligence and platforms, but it is not the same as any of those.
JN: Where does
surveillance capitalism go from here?
SZ: Surveillance capitalism moves from a
focus on individual users to a focus on populations, like cities, and
eventually on society as a whole. Think of the capital that can be attracted to
futures markets in which population predictions evolve to approximate
certainty.
This has been a learning curve for surveillance
capitalists, driven by competition over prediction products. First they learned
that the more surplus the better the prediction, which led to economies of
scale in supply efforts. Then they learned that the more varied the surplus the
higher its predictive value. This new drive toward economies of scope sent them
from the desktop to mobile, out into the world: your drive, run, shopping,
search for a parking space, your blood and face, and always… location,
location, location.
The evolution did not stop there. Ultimately
they understood that the most predictive behavioural data comes from what I
call “economies of action”, as systems are designed to intervene in the state
of play and actually modify behaviour, shaping it toward desired commercial
outcomes. We saw the experimental development of this new “means of behavioural
modification” in Facebook’s contagion experiments and the Google-incubated
augmented reality game Pokémon Go.
It is no longer enough to automate information
flows about us; the goal now is to automate us. These processes are
meticulously designed to produce ignorance by circumventing
individual awareness and thus eliminate any possibility of self-determination.
As one data scientist explained to me, “We can engineer the context around a particular behaviour and
force change that way… We are learning how to write the music, and
then we let the music make them dance.”
Circumventing: Avoiding
This power to shape behaviour for others’
profit or power is entirely self-authorising. It has no foundation in
democratic or moral legitimacy, as it usurps decision rights and erodes the
processes of individual autonomy that are essential to the function of a
democratic society. The message here is simple: Once I was mine. Now I am
theirs.
JN: What are the
implications for democracy?
SZ: During the past two decades
surveillance capitalists have had a pretty free run, with hardly any interference
from laws and regulations. Democracy has slept while surveillance capitalists
amassed unprecedented concentrations of knowledge and power. These dangerous
asymmetries are institutionalised in their monopolies of data science, their
dominance of machine intelligence, which is surveillance capitalism’s “means of
production”, their ecosystems of suppliers and customers, their lucrative
prediction markets, their ability to shape the behaviour of individuals and
populations, their ownership and control of our channels for social
participation, and their vast capital reserves. We enter the 21st century
marked by this stark inequality in the division of
learning: they know more about us than we know about ourselves or than we know
about them. These new forms of social inequality are inherently antidemocratic.
Stark: Severe, rigid
At the same time, surveillance capitalism
diverges from the history of market capitalism in key ways, and this has
inhibited democracy’s normal response mechanisms. One of these is that
surveillance capitalism abandons the organic reciprocities with people that in
the past have helped to embed capitalism in society and tether
it, however imperfectly, to society’s interests. First, surveillance
capitalists no longer rely on people as
consumers. Instead, supply and demand orients the surveillance capitalist firm
to businesses intent on anticipating the behaviour of populations, groups and
individuals. Second, by historical standards the large surveillance capitalists
employ relatively few people compared with their unprecedented computational
resources. General Motors employed more people during the height of the Great
Depression than either Google or Facebook employs at their heights of market
capitalisation. Finally, surveillance capitalism depends upon undermining
individual self-determination, autonomy and decision rights for the sake of an
unobstructed flow of behavioural data to feed markets that are about us but not
for us.
Tether: Chain
Rely: Depend on
This antidemocratic and anti-egalitarian juggernaut is best described as a market-driven coup from
above: an overthrow of the people concealed as the technological Trojan horse of digital
technology. On the strength of its annexation of human experience, this
coup achieves exclusive concentrations of knowledge and power that sustain
privileged influence over the division of learning in society. It is a form of
tyranny that feeds on people but is not of the people. Paradoxically, this coup is celebrated as “personalisation”,
although it defiles, ignores, overrides, and displaces everything about you and
me that is personal.
Juggernaut: A juggernaut is a very large
truck. If you describe an organization or group as a juggernaut, you are
critical of them because they are large and extremely powerful, and you think
they are not being controlled properly.
JN: Our societies seem
transfixed by all this: we are like rabbits paralysed in the headlights of an
oncoming car.
SZ: Despite surveillance capitalism’s
domination of the digital milieu and its illegitimate power to take private
experience and to shape human behaviour, most people find it difficult to
withdraw, and many ponder if it is even
possible. This does not mean, however, that we are foolish, lazy, or hapless. On the contrary, in my book I explore numerous
reasons that explain how surveillance capitalists got away with creating the
strategies that keep us paralysed. These include the historical, political and
economic conditions that allowed them to succeed. And we’ve already discussed
some of the other key reasons, including the nature of the unprecedented,
conquest by declaration. Other significant reasons are the need for inclusion,
identification with tech leaders and their projects, social persuasion
dynamics, and a sense of inevitability, helplessness and resignation.
Ponder: Deliberate
Hapless: Unfortunates
We are trapped in an involuntary merger of
personal necessity and economic extraction, as the same channels that we rely
upon for daily logistics, social interaction, work, education, healthcare,
access to products and services, and much more, now double as supply chain
operations for surveillance capitalism’s surplus flows. The result is that the
choice mechanisms we have traditionally associated with the private realm are
eroded or vitiated. There can be no exit from processes that are intentionally
designed to bypass individual awareness and produce ignorance, especially when
these are the very same processes upon which we must depend for effective daily
life. So our participation is best explained in terms of necessity, dependency,
the foreclosure of alternatives, and enforced ignorance.
JN: Doesn’t all this
mean that regulation that just focuses on the technology is misguided and
doomed to fail? What should we be doing to get a grip on this before it’s too
late?
SZ: The tech leaders desperately want us to believe that
technology is the inevitable force here, and their hands are tied. But there is a rich
history of digital applications before surveillance capitalism that really were
empowering and consistent with democratic values. Technology is the puppet, but
surveillance capitalism is the puppet master.
Surveillance capitalism is a human-made
phenomenon and it is in the realm of politics that it must be confronted. The
resources of our democratic institutions must be mobilised, including our
elected officials. GDPR [a recent EU law on data protection and privacy for all
individuals within the EU] is a good start, and time will tell if we can build
on that sufficiently to help found and enforce a new paradigm of information
capitalism. Our societies have tamed the dangerous
excesses of raw capitalism before, and we must do it again.
Tame: Domesticate
Raw: Chilly, violent, primitive
While there is no simple five-year action plan,
much as we yearn for that, there are some
things we know. Despite
existing economic, legal and collective-action models such as antitrust,
privacy laws and trade unions, surveillance capitalism has had a relatively unimpeded two decades to root
and flourish. We need new paradigms born of a close understanding of
surveillance capitalism’s economic imperatives and foundational mechanisms.”
Much as
we yearn for that: As much as we like it
For example, the idea of “data ownership” is
often championed as a solution. But what is the point of owning data that
should not exist in the first place? All that does is further institutionalise
and legitimate data capture. It’s like negotiating how many hours a day a
seven-year-old should be allowed to work, rather than contesting the
fundamental legitimacy of child labour. Data ownership also fails to reckon with the realities of behavioural surplus.
Surveillance capitalists extract predictive value from the exclamation points
in your post, not merely the content of what you write, or from how you walk
and not merely where you walk. Users might get “ownership” of the data that
they give to surveillance capitalists in the first place, but they will not get
ownership of the surplus or the predictions gleaned
from it – not without new legal concepts built on an understanding of these
operations.
Fails to
reckon with: Fails to consider, to realise.
Glean: If you glean something such as
information or knowledge, you learn or collect it slowly and patiently, and
perhaps indirectly.
Another example: there may be sound antitrust
reasons to break up the largest tech firms, but this alone will not eliminate
surveillance capitalism. Instead it will produce smaller surveillance
capitalist firms and open the field for more surveillance capitalist
competitors.
So what is to be done? In any confrontation
with the unprecedented, the first work begins with naming. Speaking for myself,
this is why I’ve devoted the past seven years to this work… to move forward the
project of naming as the first necessary step toward taming. My hope is that
careful naming will give us all a better understanding of the true nature of
this rogue mutation of capitalism and contribute to
a sea change in public
opinion, most of all among the young.
Rogue: A rogue element is someone or
something that behaves differently from others of its kind, often causing
damage.
Sea
change: A sea
change in someone's attitudes or behaviour is a complete change.
The Age of Surveillance Capitalism by Shoshana
Zuboff is published by Profile (£25). To order a copy for £22 go to guardianbookshop.com
or call 0330 333 6846. Free UK p&p over £15, online orders only. Phone
orders min p&p of £1.99
Cap comentari:
Publica un comentari a l'entrada