before we get started I'd like to take a
moment to warn you that the second half
of my talk contains violence and graphic
imageries of natural disasters so before
we begin I'd like to start with a PR
video I saw recently it's a little long
so bear with me
need some sound on this new feature they
want to be on the cutting edge of the
technology that's coming out so on
snapchat there's like this snap shows
there in snapchat streaks and the
morning just sent a strange picture and
when I get home from school send a
shrink picture and now it's like a habit
to do when I get on my phone when I post
things on social media there's
definitely a period where I'm checking
who saw it how many views you get and
what people are liking all my stories in
what I should keep posting there is a
lot of pressure to present a version of
yourself that's close to perfect I
almost never post a picture that hasn't
been like touched up in any way on
Instagram I always check it if it's not
getting as many likes as my other
pictures I might delete it
[Music]
I have been washed I think multiple
shows I watched about 10 to 12 episodes
in a day with 15 seconds between each
episode definitely makes you feel like
you have you have this urgent choice you
have to me instead of having to wait for
an episode to come out every week
Netflix as a whole makes it a lot easier
to just consume so much media all at
once in a row I'm still trying to decide
but it starts playing in UPS episode I'm
like oh it won't hurt you find yourself
like two hours later and you're still
watching it and you have homework to get
done I watch a lot of YouTube probably
more than Netflix and the suggestion
videos help me like subscribe to new
people I will watch videos for like an
hour and a half without even having a
plan to do that
there's just so much content that's just
addicting
this video of testimony by tech addicted
teenagers was used for the launch of a
new program called the center foot that
humane a new agenda for technology in
San Francisco earlier this year the firm
working on this concept is the center
for humane technology it's co-founded by
Tristan Harris a former Google ethicist
aza Raskin from Mozilla amongst others
in their words app addiction social
isolation outrage misinformation and
political polarization is part of one
big interconnected system that they call
human downgrading which they consider to
be a technological existential threat to
humanity through the exploitation of our
weaknesses they are members of a growing
chorus of tech executives design
practitioners academics journalists and
even people here in his audience all
calling for an industry-wide
rehabilitation one of the ways in which
they call for this is through the
inclusion of design ethics in our work
today I will argue that design ethics is
not the answer
instead design ethics is a form of
reductionism that allows designers to
escape scrutiny of their work as it
fails individuals communities and people
grapple with the realities of broader
social technological and political
realities before get into that I'd like
to talk just a little bit about how I
got here thanks to an apple developers
for education scholarship during my
studies my first years of the designer
was spent during the early app gold rush
I was lucky enough to work when teams
and as a freelancer and we were
consistently working on big products
where directives were to design
trustworthy frictionless interfaces but
my minor background in software
engineering helped me understand
understand how insecure
these systems are this was of course
compounded by the lack of
standardization of technologies such as
HTTP for secure connectivity at the same
time my anxiety was compounded by
constant international headlines of data
breaches surveillance and technology
enabled abuse
if we were to design trustworthy
interfaces I thought then maybe
encryption and privacy should be at its
core too in 2014 I prototype signal with
open whisper systems through a github
collaboration with their den lead iOS
developer fred jacobs eventually after
months of working together I traveled to
Hawaii in its last weeks to help ship
the final version to the App Store
we worked diligently diligently to
prototype new design performance
security and clear interfaces at a time
around iOS 7 when messaging apps were
either not encrypted or had deeply
flawed interfaces it's kind of
interesting every soft and this is
jarring this is a shot from house of
cards and seeing that work just pop-up
is like always like a bit of a holy hell
shortly afterwards and after almost two
years as creative director at the early
cryptocurrency startups coin jar I
joined spider urgh
a secure cloud storage company name
dropped by Edward Snowden himself there
as chief design officer
I led deep collaboration between design
and cryptography developing new methods
for trust and key collaboration informed
by our research and research conducted
by Carnegie Mellon University it is
obviously within hindsight that this
wasn't the complete answer
although the need from security and
privacy unquestionable that breaches and
have are the headlines of breaches and
abuse continued in spite of the
availability of these tools and an
entire emerging ecosystem of security
related products in 2017 I joined
tactical Tech in response to this hoping
to understand why these projects haven't
completely succeeded last year I penned
two essays based on my research the
first protocols and governance explain
examines how the role the design of
decentralized infrastructure influences
the hierarchies of the communities they
spawn for example BitTorrent the
decisions made by Bram Cohen in the
system eventually led to the creation of
what CD an enormous music underground
service return itself offered no
communication features so these
communities had to organize themselves
and at one point what CD before it shut
down had the world's largest music
collection
but while BitTorrent allowed a global
clandestine music culture to flourish
the protocols design also required his
participants we identify a
so files could be discovered and
distributed this destroyed thousands of
Lights as multinational media
corporations were able to leverage
existing and new legislation to identify
and target users not just for
downloading content but distributing it
leading to fight like financial
consequences that were astonishing this
is an example of the focus of my second
essay weaponized design a term that
describes systems and interfaces that
harm users while performing entirely
within their design unlike dark UX
patterns where unethical practice
employs cognitive tricks - in to
influence people that interact against
their own interests weaponized design is
insidious and opaque the bittern example
is one but here are others on the left
facebook's year-in-review juxtaposes a
dead child and a burning apartment
against a festive end of year
celebration
that's an algorithmic selection on the
right amidst a messaging client orders
suggest positive responses to deep
personal loss almost all popular
definitions of design ethics consider it
as practice base a responsibility of an
individual team to assess their
motivations on what they choose to build
this is born from the individualism
of our understanding of technology today
this intent here is considered key and
the scope of responsibility extends
perhaps to a larger design team or
requires that the designer is obligated
to call out unethical business decisions
consent in technology is drawn from
inclusive and progressive politics and
sets a foundation to advocate for an
empowering individual in building
consent 'full tech the wonderful
designers Erna Li and dong - Livia can
define consent as freely given
reversible or withdrawn anytime informed
and transparent it is a framework for
ethical data use policies a collection
of personal information readily given
without coercion or trickery and in a
way that can be destroyed if necessary
such ideas of consent are even used in
things like the jawed the gdpr
Europeans the European Union daughter
protection regulation in 2017 the
average smartphone user in Europe or
North America had 80 apps installed on
their device each with their own
important you
cases behind me you see a video of an
emergent technology we're banking
creditors kind of analyze social media
networks in order to determine whether
an emergent middle class is able to
accept credit
it's eligible for credit expecting a
user each of these 80 apps has their own
important use cases their own living
Terms of Service and interact with
broader networks such as these expecting
a user to maintain an agreement with
just one is the holy unknowable system
such as a social media network is
impossible informed consent is not
possible in a world saturated with
platforms and stakeholders oops sorry
going backwards not forwards consent in
the public private square is especially
non-negotiable and one directional and
often leads to abuse of personal data
and weaponized design some instances
kind of shroud the power structures and
are playful in their in the way that
they execute and get consent but others
are more serious and immediate disabling
location services is withdrawing consent
but sharing your location with someone
you trust
while on a date is a key safety
technique for people on dates in who are
vulnerable location services also help
people with accessibility or mobility
difficulties to opt-out is to reduce
your quality of life or threaten your
physical safety design ethics also often
frames its criticism from a solutions
based perspective a problematic behavior
is observed identified as caused by
technology and then corrected via a
solution the criticism of early facial
recognition technology stands as perhaps
one of the most troubling examples of
popular solution ism in the last five
years early biometrics learning with
homogeneous data sets were unable to
detect or respond to non-white features
and popular discourse rightfully seized
upon that racism embedded in these
technologies and this was in a mistake
this was a mistake that failed to
recognize and respond to the
authoritarian opportunities offered by
biometric recognition as a result these
technologies are normalized for example
Amazon girl in which you can walk in pay
for stuff and buy stuff without a
cashier through by
tricks but another video here but I
worked on to a completely technology
called building blocks is implemented in
these a three and a slack refugee camps
we authorize the transactions using iris
scanning or Irish biometric technology
the iris scan actually triggers the
private key of each beneficiary so we
use that for cross-checking but also
other services like the WFP
supermarket in this case they could go
to one of the supermarkets in the camps
and giving their entitlements you go and
buy something when you need it at the
time that is suitable the Roma somebody
upstairs in our office is Somalia or
Radha Kunda time the cashier little girl
all the cool night - here Salafi and
infogami Buffy nearly
I mean instead of layin over there so
what I had in there you know you at all
what you see here and this fascinates me
is that these two systems are really
similar their suits so two
socio-political worlds converging
enabled but importantly not caused by
technology the key difference is consent
and criticism of the technology itself
it's hard not to imagine Amazon go
expanding into the east coast of the
United States into an area that is a
future casualty of a climate change
event such as a superstorm what would
the end result of an Amazon Girl store
reconfigured to to respond to crisis in
a similar way to the World Food Program
example here I think a best example of
this of a biometric facial recognition
technology is the protests in Hong Kong
over the last six months something
that's captivated the world for a long
time well what's interesting about this
is this the disturbia nor the the
cyberpunk aesthetics around this kind of
work
it couldn't the confronting images of an
immunity in numbers against the total
surveillance absolute adversary this is
the maturity of biometrics in action an
authoritarian tool wielded by an
existing established power ethical
facial recognition criticism is a total
failure and the only defense left is to
physically tear it from the earth
in this video a woman in hit is hit and
killed by a self driving uber this
horrific incident is described as a
software malfunction but this and later
discussions of self-driving vehicles are
they being unable to recognize people
raises uncomfortable questions about
machine learning and safety beyond the
obvious cars are an intensely political
landscape altering technologies they
reshape every part of the society from
their isolating interiors to landscape
altering roads to the global supply
chain that maintains them but autonomous
cars are different for the first time
cars must adapt to their environment
through software decision-making perhaps
the most famous example is the public
research into the autonomous trolley
problem that is machine led
decision-making about who to sacrifice
an emergency based on their demographics
the question catapulted the ethics of
self-driving cars into the public
imagination but to date there is little
available research that examines or
explains how a car might behave in
systemic catastrophe we're on the
precipice of a climate disaster yet
little public research exists that it
describes how a self-driving car can
possibly navigate a situation of data
chaos or the world it is an upside-down
chessboard similar to how fly-by-wire
technologies and airplanes disengage
during emergency today it appears that
human driven cars offer the best chance
of survival in a crisis as this question
looms it is masked by discussions of
software bugs and the Machine so-called
moral compass we are yet to even begin
the discussion of the feasibility of
this technology in an era of violent
ecological collapse the death of a
cyclist is but a symptom of a larger
core problem in January its fortifies
design team launched a practice burkas
framework for ethical design and an
event in London a blog post described it
as this whether it's because of data
breaches the alleged addictiveness of
screen
or social media platforms getting caught
up in political issues trust in tech is
at an all-time low companies and brands
are forced to reconsider strategies and
place uses at the center of everything
they do in other words companies are
forced to design ethically to bake
ethical design into everything they do
Spotify is ethics framework reflects the
zeitgeist of our time but this is not
the criticism leveled at Spotify itself
the services algorithmic curation is a
relentless economic pursuit to drive the
cost of music labor to zero and this
plays a central role in destabilizing
music culture this accusation has been
made in internal industry revenue
lawsuits but artists are speaking up -
in a piece for The Guardian collaborator
of Holly Herndon mat DRI Hurst also an
artist and outspoken critic of music
platform writes about this a service
such as Spotify explicitly D prioritizes
music provenance decomposes the album
and threatens to displace criticism as a
source of music discovery you could be
forgiven for wondering if the
elimination of the very institutions
that lent credibility to the concept of
independence is a core design priority
Spotify has a world class design and
engineering team and without them the
company would not fail if good design
input is possible without resorting to
the tactics of a used car salesman and
it is then spoil it by Spotify its own
standards they have practiced ethical
design but positioning design ethics as
a practice based framework this liberate
s' the team from the problems that their
work enables and it's hard not to be
cynical and interpret this a deflection
of deep systemic problems early 20th
century designers I think got this this
is an example of this this is the think
small campaign commissioned by
Volkswagen at the end of the 1950s to
reinvigorate their company in the ashes
of the Third Reich this car sold to
American audiences reinvented not just
the car culture in the United States
creating a new category but also
advertising together altogether today
design based on philosophies built in
the first in the first calm era advocate
for frictionless mindless designs such
as Steve crums book don't make me think
this is a form of cultural collapse and
when combined
with universities that prioritize quick
and easy design degrees and training in
order to get started in code and design
on platforms what we end up with is a
remarkable set of tools for generating
new capital and products at this expense
of interdisciplinary analysis of things
at scale design ethics is a reaction to
shocking disturb years amplified by
technologies in presenting for an
addiction polarization electioneering
and manipulation as caused by technology
we elevate its roles at the expense of
other systemic factors for addiction and
kids can be caused by design and
algorithmic editorial content but it
could also be a symptom of the precarity
of family work and a lack of parental
guidance due to juggling time
constraints in dublin Jobson and
precarity live streamers could be
monetizing validation and gaming their
fans on lines for likes and views but
this is also a symptom of emerging
unstable middle-class celebrity swatting
gamers that's the practice of trolling
live stream gamers by sending police to
their house based on false reports it's
a terrifying and outrageous trolling
spectacle but perhaps this is the most
visceral imagery of social isolation and
police militarization beyond the issues
of YouTube autoplay distracting kids
from their homework and boomer parents
sharing conspiracy theories for likes on
Facebook the lived experiences and the
struggles of our users are at odds earth
is how infrastructure is conceived and
built and how much importance we as an
industry place on the outcomes
themselves as we end this decade we must
begin to look at this with a new
understanding that our work is a greater
whole
a part of a greater whole
amplifiers and not always causes
tremendous upheaval in our lives only
then will we be able to respond with
true meaning our emergent technologies
blockchains
artificial intelligence mixed reality
all hold immense potential but today we
risk repeating the mistakes of previous
generations but this time in a more
precarious global environment with more
powerful technologies design ethics no
thanks
thank you very much
[Applause]
[Music]
you