[Applause]
the difference of disciplines expertise
in the room who here is a designer or a
researcher also dude does something he's
done research before or tried it great
it's okay
all right good so you got some some
vague experience in it are you all
working on it small to use on new
products yeah okay well that's great
that's good to know okay so this the
sister unit so we're going to talk about
what we use the research is definitely
what it is not to try and glass some
myths why you should do it especially
was going to give us some information
really that about cognitive biases and
how how we work as humans especially we
do our research and talk to users as
well so what kind of what might be going
on in the users mind at the same time as
you interview them well give you tips to
doing it yourself afterwards and
depending on the output site in the room
we may focus more attention on that so
this isn't you know what we could teach
you also get all these take parts and
research with us at the end of purpose
Vasiliy this isn't a session on how to
do a usability
you're not going to come away from this
where they templates or questions to ask
people the reason for that is it doesn't
exist that anyone who gives you that is
not doing their best work it's always
different people are messy this is about
helping you get the skills the
confidence to try to ask better nitrogen
to get better results out of it so what
we ask is that jamison workshop that we
wait to participate fully in the
activities i learn to keep an open mind
and interrogate your own ways of
thinking as well so just a little bit
about us i'm georgia raucous and i
design research immediate consensus with
a company for a year
DEFCON markzware bursaries so it's nice
to be here again and i do use a
researcher about seven years now i works
at number different ecommerce financial
government projects and before that
would Europe's leading usability testing
company as well a consensus I work
across lots of different spokes helping
do research at all the different users
including with Sasha and I also help
train designers to do research as well
so hi I'm a special message
right now I have a designer event
different senses and enviable there
[Music]
I shine all the time user research and
this is why sometimes I
nice Bernie's think my opinions about
users and why they have their own right
okay so there are actually enough
people's do this to pass which is a bit
better than race so you can just grab
somebody just to find out a little bit
about them of their experience of doing
research afterwards to each other
eight years
-
[Music]
as I have experiences with doing a
one-year system did some of you already
know that is that up - yes I did
another industry
set up a system what Dan was really
growing in Dan's money
get on that my Expressionism NBA's
summer program
the major Association is America
well she's really American poets hear us
graduation
[Music]
projects
thank you it's my appreciate I just love
tonight if anyone would like to share
any any of their biggest pains doing
user research or the biggest problems
that they've encountered and then what
we could do we can try and focus on that
towards the end of the session it's a
Taylor series so it's anyone got
anything a dying to know about I think
you said that you were interested in
finding people to talk to how to go
finding users anyone else with anything
else that they're interested in all of
it amazing okay cool
all right so what is user research we're
not going to go into too long about this
you've all got a sense of it but just
want to explain this is Margaret means
she's a very famous was a very famous
anthropologist and she famously wrote
this what people say what people do and
what they say they do are entirely
different things and she discovered this
by observing people in the natural
environment and interviewing them
separately and doing surveys to train
loads of different methods this was in
the thirties and it was quite
influential at the time and also quite
would come true controversial too
because this was around the time where
man was meant to be master of his mind
right so so what consumer behaviour the
science around that has proven this to
be the case so people not very
insightful about their own
rather habits behaviors motivations they
need to help eliciting that out of them
and that's the job of the use
interviewer so what is user researchers
we're talking about qualitative research
here you could go and do big data
surveys we'll talk about that in a
minute
but it's usually interviewing or
observing real people and what we're
looking for is identifying these
behaviors goals motivations pains all
that good stuff which tells you what you
should be building and understanding the
why behind the what they do right what
they do is interesting but it's hard to
make informed decisions based on that so
what we want to decide is what is the
intrinsic motivation behind people's
families and then the whole purpose of
research is to have an action plan off
the back of it
right you can't be seeing these paths
and themes across users and now what
you're going to want to do is come up
with some recommendations and actions
and why should talk at the end about how
you can you how you can help do that
with your team as well so what user
research is not it is not about people's
opinions it is not about whether they
would prefer the yellow object or the
black objects right that it does kind of
market research it's as well market
research is about asking many people the
same set of questions and actually you
ended up learning a little about a lot
of people the benefits of qualitative
user research interviews is you will
learn a lot from a small number of
people and data analytics you know site
usage
what's happening with your product that
is very interesting what data ok but
there's two problem is first thing the
project
is that most apps or doubts or services
don't have many users so you don't have
data then how else you do find out what
people are doing a lot of companies are
actually not intrinsic intentionally not
building it any analytics products into
their product of the basis that they
want to keep it pure I'd like to make
sure that the users behaviors are not be
tracked and things like that by any
third party like Google Analytics so
data analytics is useful but you need
like a lot of users for that to make
sense otherwise you're basically just
looking at what all your development
team is doing on your website and it's
not something which I I've experienced
before I thought we've had a hundred
users 70 so something you only do once
you build the thing right it's this is
the wrong time to be doing research we
need to be doing research at the start
of the thing really before we build
anything there is a bias towards action
and a bias towards technology within
blockchain within the ecosystem which is
let's get building something okay but I
question the value of building the one
thing I really like to encourage people
just to at least do some research to
start things off yeah so why is being
able to interview users important to you
you probably heard the statistic
somewhere between 50 and 90 percent
depending on what article you read
between 50
90% of all startups fail in their first
year right and then the remaining last
start failing and only a small portion
last two year five okay so based on that
information most people in this room are
working on a failing product and one of
the people just to think about that for
a second right and like let's let's be
open to this because we we live in a
world where we have to with cheerleaders
for our products all the time we're in
our own little echo chambers and our
bottles are yeah it's gonna change the
world well we don't know that to be the
case and the way we could do risk is
doing research early so just get some
done first right before we build the
product to find out it's the wrong thing
and that's the whole point of saving
time and money by doing this it could
really help align people around the
problem as well that they might have a
worry in this early stage of this
pioneering technology and deserting the
way the people of businesses interact
with one another but because crypto got
change the world is relatively small and
highly technical is easy for us to think
that our users are just like us right oh
we're building a product who's the
product for those people liking okay
that might be true but it's get is a
guesswork it's guesswork until you're
gonna find it home right so I'm not
saying that you're wrong
but maybe just going about verify that
because not everybody speaks in the same
languages us and I mean like technical
language as well as nationality and
other language as well so that's really
important and probably you don't have a
dedicated user researcher on your team I
there are many most teams don't have
someone like meeting your session to
come and do all the research for you so
don't just leave it until you you know
six months down the road and then you
are you your designer then you get them
to do some research you do it first I
think it's a really great of an
empowering activity so I can I know that
is a still faster tech failures
okay so how many on product many how
many of you have done user research for
product not so many it's okay you're not
alone many tech companies forget to do
that and they ignore user research and
this is called tunnel vision and that's
like the deadly scene design ignores
users and users if users feel that they
are ignored by design
[Music]
design so I will show you some
heartbreaking testimonials
so this is from a crowd they say we
spend way too much time building it for
ourselves and not getting feedback from
prospects it's easy to get tunnel vision
I'd recommend not going more than two or
three months from the initial start to
get in the hands of product and prospect
so if you are working on your product
and it wasn't liked more than three
months it's alright you can feel it now
and this is from Voltaire type we didn't
spend enough time talking with customers
and we were rolling out features that I
thought were great but we didn't gather
enough input from clients we didn't
realize it until it was too late it's
easy to get tricked into thinking you're
your thing in school you have to pay
attention to your customers and adapt to
their needs so probably we all have been
there
[Music]
so what does bad user research look like
okay so the first time provide user
research is no news of research you may
be familiar with this quote from Henry
Ford got a question mark there because
actually the first attribution today was
in 1970 this quote so who knows we
actually did say so that I've heard this
from people in love chain as the reason
not to speak to customers okay there's a
few problems with the history it's also
like it's used to argue that like true
innovation from comes from signaling
gifted individuals who know better than
anybody else right I'm sure we all we
all know people who kind of behave it
like that but the problem with using
this quote as an argument is that this
is not user research so what he's
actually talking about if I had asked
people what they wanted we don't ask
people what they wanted user research
that's capturing opinions people are
very poor predictors of their future
behavior okay so this is this is just
bad research that he's talking about
doing okay in addition Ford the most car
company
due to their lack of innovation so so
they obviously created great products
but then all the innovation that
happened after that was about the
production line to save money to build
cars cheaper you may remember you it's
also apparently said that you can have
any color you want except black so what
happened was for the cars motive the
share of the market dropped from 66% in
21 to 15% in 1927 six years a massive
drop in market share so you can use that
as a fact next time that somebody says
that we don't need to speak to customers
the competition was innovating in the
space and Ford was not and that's the
reason why that happened
maybe after today's workshop you will
think twice about putting this out okay
I look we've all done this at some point
right I made a thing please give me
feedback yeah that would be amazing so
what happens what sort of feedback do
you get who answers everybody people who
are your fans are these people who
already know what your product is right
you're not really what about people who
refuse to click on it maybe you should
find out what they think because also
talking to detractors is really really
valuable and then this brings us on to
surveys so most surveys are absolute
garbage
I'm really sorry to explain this to you
now there's a few reasons why this is
the case they feel like think they're a
good tool that's because they do make
good sense every company is putting them
out there all the time you always get
requests to fill out a survey right so
by frequency it brings validity the
problem is is they're very easy to
create a bad survey survey design is
actually a science right they're
statistics that goes into it in order
for it to really help you you need to
think put a lot of effort into it but
it's really easy just to throw one up or
what should we ask them what you liked
what you dislike lalala send it out okay
and I think
is often used in instead of user
research because talking to people is
scary right so it's like what the survey
to do the talking for us and we expect
the survey also for everybody to fit
into these neat boxes that we've already
determined for them but people are messy
and Amorphis okay so if so it's not as
simple as that
and I'd like to just change this a
little bit further it's also to think
about your own use of surveys so when
you get sent a survey do you fill it out
why do you why do you not is it because
you have a particular relationship to
that brand what happens when the survey
is too long do you abandon it halfway
through what happens when you see a
question that you're not really sure
where it's asking you and you don't feel
that surveys asking you the question you
really wish it was asking you and then
you get asked to rate something between
1 and 10 like what's the difference
between a 7 and 8 someone's actually
gonna do some number packaging around
that they haven't thought about the
science of it either right so they're
not that useful ok I hope you put that
across and also we are default to we
have confidence in large sample sizes
right so we think that them oh good well
it's better than speaking to a few
people when you've got loads of survey
responses and now give us confidence
about the answers that are there so
firstly if you're how do you get large
responses a big question around that but
loads of surveys where you get like 15
responses doesn't really help you make
decisions B question around that and
also if they were to see is only true
that they are give you more certainty if
you're doing a robust recruitment method
to recruit a very large group that
represents the people in your users and
then randomly select
from them that's the way to have
accuracy at large data no one does that
when they send our surveys they just
blast their customer base and hope that
some people reply so we guarantee such
your I put like put money on it that if
you go to speak have deep interviews
with five people you will learn a lot
more than you will surveying and get no
response for twenty people I promise
okay so another type of bad user
research is doing it with those who are
easiest to access the people who love us
they're great they're like our
cheerleaders they want us to do really
really well alright the problem with
this is that they're probably too close
to the products they may speak the same
language as us in terms of the
terminology that we use for things they
probably have an idea of who you are and
they want you to do well so they're
probably not going to be very honest
about it's okay and and and but there is
a there is a use case for doing this
practice your interview with them
they're great people to practice
questions with so don't feel scared to
do it it's just these aren't your users
ok these this is just other people's
opinions as well so so yes I'm going to
hang status to talk about them
brain hacking yes so what we were
that you know
brain hacking we really get to learn
down our greatest vulnerabilities and
how to turn them into opportunities and
into to put them on to our advantage so
I will run you through some cognitive
biases and feel free to raise your hand
if you spot them and you have
experienced them so what are cognitive
biases they are mental shortcuts our
brain thing this means that they are
unconditioned and we can't avoid them
but what we can do is learn how to spot
them by knowing more about this matter
and because cognitive bias is loving us
all also researchers are there winking
so why it's so important to know about
them is because as researchers we we
might spoil our our world so let's let
me give you an example you are doing
research you have some assumptions you
are superb like you you're super super
thinking that that assumption and
because you want to really prove that
you are formulating your questions like
you know just a program so you are
influencing and biasing your user by
articulating
not very well the questions so but not
only researchers are the only ones
affected by biases users are humans too
so there you have to bear that in mind
and when you do user interviews don't
take their words like for granted just
try to attend tonight so we'll start
with the response bias and I'm quite
curious how many of you have experienced
it I handed myself many times the
response wise is
it's caused by the way we collect data
this means that you can bias your entire
research by the way you articulate the
questions sometimes this happens when
the development team conducts the
research it's very hard to step away the
objective
especially the responses you are getting
are not what you want to hear
have you experienced experienced it have
you went through that domain put your
hands up okay a very severe drought I
love it
[Music]
like everyone who works on the product
this is the development Ian I'm not
showing okay so get into our next bad
boy
this is the desire for harmony in a
group which results in a dysfunctional
decision-making we want to minimize
conflict and reach consensus so we might
suppress opposing viewpoints and isolate
our team from outside influences so we
love our product so much we don't want
to hear anything like outside
confirmation bias
this is the tendency to search further
interpret focus and remember information
in the way that confirms our beliefs so
this happens when you do user research I
don't know you only hear the answers you
want to hear so even though 95% of the
users that use something else you only
want to hear the 5 percent so you get
some supervised information that that
happens the million Alessio and we were
working on our blockchain Explorer we
are so convinced that our core users
were just power users so we only wanted
to hear that curse of knowledge effect
when better informed people find it
extremely difficult to think about
problems from the perspective of
lettering form
people this is what we don't really use
we we use the terminology and think that
everybody should really understand about
the think of it you're you're working in
blockchain right and you talk to
everyone with the terminology and expect
them to really know what you are saying
but it's not all right you don't have to
make your user feel dumb don't challenge
your user even if they are an expert
[Music]
social desirability bias yeah I mean
this is actually very you can encounter
it all the time thing when you're doing
the research but I'm against blockchain
me who can you exploit now here you go
to rekall buicks and if you have a user
who has never heard of go chain you
confront them with play all the apps
onboarding
we'd make something sense this new user
they have no clue what I don't know
yeah that's alright nothing given I
think that's one of the biggest
challenges for us and what changes if
you have people that know who
okay let's make it successful I think
this is our role to do education and
social desirability bias is about users
you interview might give you the answers
they think you'd like to hear over there
true beliefs because they want to plead
you so this always happens to me and in
order to avoid it when I do a user
interview I tell them I never worked on
this sign don't worry about the people
who are working on these are not part of
their research so you use them in so you
make them feel more comfortable and be
honest
Hawthorne effect this is when people
know they are being observed they might
change their behavior they might think
they are being tested in an interview
instead of just listen to so you tip for
this to be to tell them all the time and
remind them every now and then during
the interview
you're not tested only the product is
tested and they're there once there once
happened to me when I was
testing with the user he didn't know
what an avi code was and I was started
to be very reluctant to answer to my
questions and I didn't want to feel bad
so you can avoid them IKEA effect I
think over designers and the development
team is affected by this we place a high
value on products we created or worked
hard for let your users do something as
part of the comporting process not too
hard but rewarding so they can connect
with your product but let's stick to
that we place a high value of products
we created every time you do a research
or you can get back because you love so
much so that will you tend to not hear
what people are telling you and but
there's also a reverse type of thing
actually you can make it make these
biases of an advantage as a researcher
or a designer you're the designated
messenger for bad news so you get the
bad feedback and you have to send tell
it to your development team and in order
for them to feel positive okay I went
somewhere I have invented this magic
they're all left out you you
invite them to take part in the user
interviews whenever they can or witness
everything that that's happening because
UX has to be transparent
we love transparency in blockchain right
so we have to make all the processes
transparent and what this is about the
fact that everyone can do user research
it's a skill that you can learn you're
not born with it and you only have to
get in the mindset of a user researcher
and that means that you have to be a
detective
think of it you're the Sherlock Holmes
of your product so you have to pay
attention to all the details
dig deep ask all the WH questions you
think you need to own in order to get
all the answers and the innocent also
think that you have to be a psychologist
you have to understand Wow how people
are working and why are they doing that
and just try to see what tickles them
and yeah and also you have to be like on
a safari you have to observe people but
not to be an active observer just try to
take a step back but to influence them
with your opinions your biases this is
not a heavy petting zoo it's just a
safari
[Music]
okay so the first thing that we're going
to do that this is where we can start
getting into activities now so who all
have your coffee and woken up so what we
do to start any research is stating our
assumptions and this is a really
valuable thing to do with your team so
the three assumptions are basically what
do we think we're going to find out when
we start interviewing people that's what
we're trying to that's what we're gonna
put down okay they don't have to be
right you don't know the answer to these
things but there are lunch there it gets
like I think probably this will happen
all right I think X because Y it's
really good to do this as a whole team
exercise so using that IKEA effect again
get people involved early then they will
feel heard and care about the outcome as
well and then also when you do as a team
exercise you start servicing where there
is disagreement where there are
conflicting views about beliefs that
become the people in the company have
those are great places to focus the
research on so if you've got a debate in
your organization about which way should
we go with this particular product
feature or whatever then interview
people can help you know solve that
question really so it states it like
this I think X because X 1 and it's fine
not to agree about them so it's just a
basic example Sasha's gonna announce two
things to you as well so an example
research question right so let's say
you're building a centralized exchange
where's the gap in the center
it changes now which could be an
opportunity for us right quite
straightforward so an assumption might
look like this I think what's missing is
a mobile app because if users can access
the decks on the go this will lead to
more usage for liquidity on the exchange
okay so Sasha's freaking out some things
here we need to get you two groups of
three so if you can organize yourselves
in groups of three please so we're just
going to spend probably only about ten
minutes on this activity and I want you
just to work together to state what your
assumptions are about the thing I've put
in front of you okay so in your group
you should all have the same thing yeah
you don't have the same question right
so how do we the research question and
just write your assumptions down you
don't have to agree with each other just
work them out okay
[Music]
which is
so trying to copy by the list at stake
that because as well right so what we're
thinking is people are going to
interview someone about this what would
we expect to find out
[Music]
the usual
I got to obsess more yes
they travel journey from the heart to
the attention and I'll be sunset
we know the forecast all users
[Music]
is what we think
all right we can have the user
perspective that I have that also we can
have a researcher protects against
yes yes power W over session will be
very of us except Kennedy that is what I
understood
[Music]
the user respect experience is your
experience
[Music]
yes
[Music]
anyway what a teenager our cars
beautiful ways that people made this a
better experience means that expects for
people so if you're gonna ask people
[Music]
experience I have this issue that
everyone else that the same machine as
well so that's a good assumption
I got every season we where it goes
first your muscles done yeah it's okay
our solution interesting
that's a weird question
it's great when your establishments I
think you felt about the expo is these
other tracks not that the
[Music]
like magic just Helen
yes
I was speaking out here when I wrote my
favorite cement food but then we
[Music]
and here it's a right away once you buy
the Wi-Fi
but otherwise I'm you have a little
mouse yeah we always be like you
[Music]
the directions so I think that the
costumes
I was more focused on building companies
to build travel for what comforts that
most basic federal powers to
commercially to this compressor customer
reaching out there partner today I'd
like to know what America yes yeah
whatever II know what you mean it is not
yes
5:27 yes
maybe
[Music]
okay last minute last assumption and I
know
and maybe you're thirsty to face the
different questions about you
yeah I must've arranged everybody you
can wrap up now some assumptions aren't
down with you brilliant so we're going
to go through now sort of the anatomy of
an incident all right
you will have in front of you some
stickers so it's usually three roles in
an interview facilitator a user and an
observer use observers really useful as
the person who can take notes the
facilitator the observe it doesn't talk
during it it's important not to have a
three-way conversation you want a long
conversation between the user and the
facilitating separate so you know you
probably guessed by now you need to
assign it yourselves with the stickers
so doing that now quickly just to stick
them on you usually be a facilitator
who's gonna be nice
okay so so I'm going to talk through
some good versus bad interview questions
oh wait right that gets not surprised
you're going to be doing some
interviewing in a minute all right
you have not prepared for this so it's
going to be uncomfortable and awkward
that's fine again it's just it's just
getting a little bit of the experience
of it okay so we've got handouts which
we're going to head out in a minute
which have all of this stuff on my cheat
sheets for how to interview people what
the questions are about questions so
good questions they're open-ended
they're things like tell me about
subscriber time tell me more about this
thing okay story time is quite good like
walking me through the last time you did
X when you started to the end but you
need to be following up with why why is
that
how come tell me more about that right
these are all very like curious
questions that allow the user to answer
from their own experience without
recommending them right you're not
telling them what do I hear they're very
much like you tell me right and then
when you hear something that's
particularly interesting you can ask why
about that particular thing so on the
flipside bad questions are closed and
with yes or no future casting everyone
is dying to ask this question would you
pay for this right you can ask this
question if you want right but the
answer you get will not reflect reality
people are very poor predictors of their
future behavior okay this is you can
quite advanced market research to to
work out what people are going to
for your product and how they'll pay for
it and whether they will pay for it and
also the best thing to do is to
understand the pain so if you actually
really want to find out someone's going
to pay for something try and find an
analogous situation where they they've
moved from a free product or paid
products in the past what made them move
you know what what was enough to
incentivize them to start paying for
something when you can understand their
past behavior then you can start
predicting their future behavior but
don't rely on them to tell you because
they will not necessarily get it right
okay I'm leaving questions so we talked
about this already quite a bit their
questions which suggests to the user
what you have in mind right and a very
simple example of this right is do you
like that do you like that is is a
leading question okay it's also a closed
question so enter the yes or a No and
it's it suggests for the user that they
shouldn't like it right then there are
some varying degrees of improving on
this question until you get to a really
good one so this is kind of a bit weird
do you like that or not
you started out balancing that I there's
two ways that could go and that this is
like really do you like that or not like
that you probably see survey questions
phrase like this trying to not leave the
user and be balanced but the best
question is this what do you think of
that allowed them to use their language
to describe what it is that they're
experiencing okay so this is the way to
get really good stuff out of Google yeah
sure
[Music]
instead of like it is nothing so you
give the option to it also the negative
but what if she specifically want to
have like positive ones but if
somebody's like always like negative
things yes what an apostle but you know
still you speak good good question so
you can do that later on so the idea is
that you start quite broad right and get
information from them and then start
going in it's a good question that might
be what was your experience of that what
do you think of that okay and then they
start going negative negative negative
negative
was there anything you liked about that
yeah what did you like about that that's
perfectly reasonable but it's just best
to staff allowing them to tell you yeah
yeah so we'd be grateful for their time
and honest with the purpose of the
research research is not the same as
selling your product and the amount of
calls are being on where the product
team is trying to pitch at the same time
as doing user testing these are not
compatible at all right when you're
pitching to a client they want you to do
the talking
they're not going to be honest about
their experiences to you why would they
do that keep this thing separate make
sure they know is this customer you're
not selling than anything you want their
feedback their their responses to the
questions that you've got you need to be
friendly but not friends when we're
friends with people and we've all
experienced this I love having great
chats with my friends right but this has
come to the social desirability bias
they you will have a wonderful chat
you'll all feel good at the end but you
probably are not being totally truthful
so it's about creating a bit of
- about honesty to be there as opposed
to just be chummy and agreeing with
everything everyone says all the time
let them do the talking as much as
possible it starts slow down here okay
let's compromise
we're influencing them and like we said
before don't receive technical or
knowledged technical knowledge use plain
language where you can that's quite
important so we're actually going to
demonstrate what human research
interview might feel like so I'm gonna
have a word to Sasha because she's going
to be so we're going to do the playing
so I will be the researcher I'm
researching Fitness out and George is my
user that I am going to interview I will
start by asking her broad questions
about her just to ease her in and find
out about a bit about her context then I
will start to ask her more questions
about
Fitness brought questions to see what
comes out and listen and pick out some
of the things she says and then start
getting narrower and asking her more
granular questions about the Fitness
Africa this is my end goal but I don't
start directly by asking her for that
because I might not get the so many site
that she might give me for I'm biasing
her somehow
she will think that oh I'm only used in
Fitness I don't want to know anything
about her or her life or anything like
that
so she will stick only to that subject
so okay
hi Georgia thank you for accepting my
invitation for the interview I am Sasha
I'm a design researcher and I would love
you to tell me a bit about yourself I'm
talking to some content about me I'm
Georgia I work in travel and travel
insurance I live in London I've got a
husband and cats
okay nice but what do you do when you're
not working so I well I work from home
so I try to get out and about as much as
possible so long lazy at home I do a lot
of reading you guys do like walking I
hang out with friends and I also go do
yoga as well
sometimes so now she has mentioned yoga
so I will speak on that okay some of you
have mentioned yoga well can you tell me
more about that yeah I go to a place
called yoga space London it's not very
far from my home and I kind of like that
maybe once or twice a week like I like I
buy a book the sessions online I just
like buy a whole bulk of tokens and then
I can book or the sessions I want so
you're using a website for that London
yoga studio you mentioned but what do
you like about the website I really love
actually the thing I really like about
it is that I can cancel the sessions
with like no hardly any notes it's just
like two hours before because like what
gets in the way a lot and I'd like
people just dump meetings and my
calendars so often have to move move my
exercise sessions around there okay and
now I'm getting narrower and where do
yoga
I don't do get a great around Soviet
studio why is that
I've tried doing at home but I can't
really do this okay not sure about space
in the house and also it's just hard to
get motivated obviously I get many big
words from a partner she doesn't have
space but they not do because I
but she also said she doesn't get
motivated so now I'm picking on that and
have you explored any ways to get more
motivation so you're hurt on that before
yeah I used to like fitness apps before
but it's really such a send us stop I
just don't really stick with them and
it's very good to ask many wise try to
find everything about what they say you
know I'll ask her why why did you stop
using that to be honest I thought like
not a very modern phone and all the apps
are just too large so answers end up
deleting them starting to understand
like some pains potential opportunities
as well okay so they recently caught a
pretty light weight so people don't have
loads of room on their phone it doesn't
get automatically deleted which is what
Apple does now
so just simply facilitates tips of the
Train actually we're going to share
these with you so the Firewise this is
root cause analysis and you may be
familiar with it it's comes from a new
battery industry I think and the idea is
something came off the production line
that's Godfather's why is that we made a
piece of machinery was working correctly
why is that management had changed why
is that but so by asking why many times
you start to get to the real reason
now obviously you're not going to ask
the user why five times okay and you
don't sound like a robot is supposed to
be this sort of a conversation but maybe
once or twice it will get to the real
reason if you remember that users don't
have this why always there at their mind
at this top level it's not readily
accessible so you have to ask it they're
not going to necessarily tell you the
reason why and that's one is the echo so
you know then you can encourage them to
say more perhaps they've said something
a lot we surely don't really understand
you could just kind of repeat it back to
them and they'll hopefully say some more
kind of on a similar thread the
boomerang so this is really good in user
testing and if song goes that they're
trying out your app and they're like
really I click this is the kind of thing
to a confirmation page and yeah they're
asking you a question so your automatic
responses to answer right don't answer
the question just ask it back what do
you think will happen when you press
that one then they will explain okay so
just so like any questions that come
back at you throw it back at them
silence I'm really bad at this that you
composite could probably tell I like
talking or tuning in
or quit silences but the more silence
you can give the more other people will
do that so they use and they will say
something that extra little nuggets and
there are many times of user research
interviews I've done where I just use a
technique where I'll just have my back
that said something they've paused I
don't just go straight to the next
question
sounds like there's something else there
so I just write some notes and just look
busy and then they just continue talking
and it's a real waste of McArthur which
is really exciting
but that does what and then this is the
question right about you know when you
when you're just not getting what you
need out of them right you use the
Columbo method alright so right okay so
would you ever use it like what what
about these fitness apps you know right
just like get me out of them right at
the end because you the purpose of the
research interview is to have some
insights okay if it's not going anywhere
that they're not seeming to get the
questions that you're asking you can ask
a more direct question but just maybe
try the other techniques first before
you get here and everything so Sasha is
giving you copies of all of this stuff
you can take away with you it uses your
cheat sheets for when you're running
user research interviews but so you're
listening for the reason behind the
answer not just the answer that they
give you
habits behaviors issues annoyances
opportunities all of that stuff is real
gold dust and in this session that we're
about to do now in the workshop we're
gonna interview one user right but
really you do more than one user so then
you're looking for themes and patterns
across those users I will will talk
about how many users and a little bit
later on so I know there's some
newcomers here hello you've missed out
on the first activity that that's fine
why I suggested that you join a group
and be an observer in what's going to
happen in the
okay so let's try it out
users you got your user stickers on yeah
okay I want you to a platinum band you
guys aren't going to stick us up and
please please open it up because you're
going to be reading around so it's easy
for people to identify it so you're not
going to be in your groups right the
user the user from each group abandon
your abandon abandon ship
you don't need your assumptions with you
move to another group okay so the users
are gonna move around
so the facilitators
Center lat
so facilitate sandals you're gonna do
some question are statements I haven't
prepared you with questions it's not
going to be easy right this is just an
experience for speaking to somebody who
probably and does not see topic that
you're going to ask questions that maybe
I thought okay so just go ahead let's do
it for 10 minutes remembers let's start
with those broad questions get to know
them start narrowing in on what you want
to find out observance please listen
carefully and note where suchan's are
incorrect or to enter okay
it's
[Music]
I would say we're working on the title
makes an int or
which is what you
it's not like that we have a conference
everything is happening so I've evolved
directly into a fight
[Music]
sometimes hours to recharge
I mean what's that feels like it's gonna
think it was hired I have to go back to
my hero
I just said with these or there's no the
compression might have to be conquered
and that's the
don't worry
in ever
[Music]
I mean
not certain that world
to be available so you don't have to be
around to watch they enough to talk
I know that
Melony
but still think the secular is the
better
yeah
by water
experience pangs in the end with people
[Music]
that's like two minutes
what
[Music]
yeah
[Music]
okay everybody thank you thank you for
so willingly getting to the rolls I
really appreciate you that's nice
so we were originally going to do this
so we were when you get back into
original groups but I don't know whether
it's to necessary I'd love for the
people I'd love to go around each group
remember to share the experience I'm
interested first of all for each team
did you discover whether any of your
assumptions were true or not what about
you guys over here and it always they're
all surprising experience
[Music]
they're like a lot of assumptions that
we wrote down and especially to be both
like the reason I did we use
and you felt like beep touch both of
them but for me that's the server I
think the six example that the
speed-dating coming in give it also your
attack reaction you're like what the
really notice W was like you so I think
that also if you're asking questions you
should really keep it like yourself and
see if you can try to that validate them
exactly this is why it's helpful not to
do it by yourself is to have an observer
gotta keep an eye because you need that
that person who's not trying to stall
the information in their head I've also
asked but not leading questions it could
be quite trying to do it here but in the
end to say like okay did we check our
assumptions I don't think we checked our
assumptions better you didn't rise in a
knowing you require a discussion guide
which is and I have the questions in it
that will help address the assumptions
right I just learned it was on you cold
so it's no surprise something everybody
writes about but well done you okay yeah
so I submit my head
what about team but did you have the
same question say makes us question
about that come attendees yeah okay I'm
done within your group and that's good
right but I mean obviously you wouldn't
then just take the information from one
you say you would do this across many
and as you start to hear the same things
that raises the confidence very high
yeah and what about over there and back
[Music]
or research question was about buying
experiences that accompany this and like
what the assumption which we didn't like
that I said everyone bought the ticket
just like here's the way let's let's act
like just the ticket page but I'll use
it actually take it from another person
former sponsor that what you think of
responsibly by the selling services this
is an example of getting out of your
problem right this is where we think
everyone's has the same experiences us
but we can be directly challenged doing
this kind of thing yes and over here and
this team yeah so what was your research
question I think you know my listen
westeros what does that experience like
for someone traveling to another country
for a world conference and what might
make that experience better
so I mean first of all something you
have experienced is that asking
open-ended question is very challenging
especially if you don't have that
experience you will tend to ask do you
would you yeah exactly
it was the over or leaving video so it's
a lot of practice and also I found like
going with the assumption also not
straightforward because the user could
also steal the conversation because
there's a second there's a you
you have some assumption but the user
could deviate so keeping the
conversation in the control aligned with
the assumption also I found it a bit yes
that excite me because there's no
Experian I mean wait yeah I mean you
you've come not prepared at all so this
interview right so in real life there
would be some preparation for this you'd
have a kind of a script of questions
they also you do have to be flexible
enough to go off-script because you know
if the users tell you something
absolutely fascinating basis I mean
follow that you know go down that rabbit
hole it's worth it's worth doing it but
yeah it's practice and practice with
people you know the last point in terms
of assumption hit rate I would say we
were like fifty percent were fifty
percent like correct about our
assumption 50 percent indeed that the
user has different different thoughts
and different opinions importantly so we
were not like completely yeah it's
pretty fine maybe politicians so I think
we it's anyone got any particular so I
think nice example with us was watch the
questions was like okay what do you
think would help you meet more people
would you like
don't I open and questions Fitzgerald in
the right direction and then directly
after was like the bias instead of like
answering the question was like the best
for example we feel the workshops are
helping so and then directly you're
feeding him basically okay out what if
you love warships so and I think that's
important like if you sort of like ask a
question and there's this silence it
might be two separate versions thinking
back to think and then that's the
Columbo question at the end right so
what about workshops do they help you
write you know and you're like trying to
find this app but to do it give try the
other options first exactly yes I think
so my language like be patient curious
based on your experience how do you
analyze that would you get here so are
you describing the observer should take
all the notes I also take notes as a
facilitator because it helps me kind of
process parts of the interview too like
I might like underline something I'll
come back to that things like that you
can we'll talk about analysis in a
minute but it's a matter of plotting
your insights across your users right so
any kind of matrix like a spreadsheet is
good for this but then also we'll talk
about this if you're we have distributed
teams so you want to collaborate things
like mural is great for that as well
right so thinking about where everybody
can see all the information especially
if more than one person is doing
the exodus right be able to put it all
in one place and actually getting the
insights into that thing is the long it
takes the longest coming to the
conclusion can be pretty quick when you
see it all there it's sometimes if I go
it's light bulb moment yes that
advantage to do it got the conversation
yes it is talked about that you may say
let's just I just want to hear from the
users what your experience was like and
then we'll have asked about questions
about how to do about this so what was
it like this all right I mean I use a
researcher so I could I could see the
Sun is still I think it was I was not
living by facilitator in any moment
maybe this specific question and then
some clothes issues also work where the
[Music]
time yeah what about you use them I
think I had a little bit of part of
additional perspective because I'm in
this session and I actually I didn't
fully process what was happening I
thought that they had the same topic
yeah and so I was waiting to get those
questions which maybe is an insight
about really users experiences they come
with opinions
time's yeah and they think they know
what you're gonna ask them and they're
excited to talk about it and then this
is like a completely different
conversation she's already the topic
great okay any other insights from users
what what was the experience like no
[Music]
okay great
so we hear a lot from people that they
want the tap play on how to do user
research right and it feels a more
personal and scientific because it's
dealing with real people people are
messy so often companies and I've worked
in some of these companies as well they
cling to tools and software user testing
platforms hoping that if they use the
right tool then they will get the right
results out of their research right and
there are loads of platforms and tools
out there for doing this stuff companies
is a building on demand right but user
research doesn't need software to be
done effectively sorry to tell anybody
here
so overtime it but in my experience my
talking as the munition a lot so I
usually use of these platformers and
stuff but now it's just quite simple so
I'm going to talk about how we do
research at consensus and I think maybe
we could just make this an opportunity
for questions and answers right as we go
through right so you might want some
tips um or ask for more information on
air as we do and so you know if you can
get the interview part right then you
really don't need to rely on any of the
other tools and software because you
know you can use you can use a user
testing platform but if you ask the
wrong questions then you get garbage
outs so it's about being able to do the
interview as well it's the first
important bit it's also the scariest bit
and the hardest bit so that's what
people like to avoid and hope that the
software doesn't for them so we'll just
show you how we do it here so there's a
first of all is about who to talk to and
where to find these people we get this
question all the time
you really need to be people outside
with your work or your bubble being
totally honest that consensus is a
really big company and there are still
some projects who only test with other
people in consensus right these are not
our end users or they might be but how
could they possibly be all of the users
you know that's pretty neat audience
that you're building for so finding
people to interview is really important
so for example say you're building a
product for dark developers like
recruiting those people at a conference
like this is a really great thing to do
it's a good way to do it because
probably there already you know it's
close to your target user and you have a
conversation with them to establish
whether they are really what the right
kind of person and then you do you don't
do the interview at the conference
people not in the mindset for taking
part user research they'll say
yeah I guess and then they won't turn up
for the thinker's only something else
more important that they want to do
right so just get the contacts make the
connection and then do follow on with it
afterwards we use I would not paid but
to promote them but I love the
respondent do also using it interviews
comm they you create the screen and you
specify the type of user that you want
they go and find those people present
them to you in a list with how qualified
they are you can check their credentials
check them out on LinkedIn they're real
people you can message them to us and
clarifying questions and it's a
scheduling tool for all your sessions as
well all right so and they process the
payments as well for the user so this is
indispensable while successfully use
this to find space enthusiasts in Kenya
soft commodities traders in Brazil that
developers people who use maker down
right you can find these people at these
platforms so give them a try it's also
free to just launch a project project
you don't pay for anybody until they've
completed it so is even you can just
test to see whether you can find the
types of people that you're interested
in talking to or just give it a go so
really all you recommend that snowball
recruiting is a method where okay you
found that perfect the perfect user
after that at the end who else should I
don't talk to it get them to do the
improvement for you right so just
passing out handling that honors and pay
them for their time there's two benefits
to this right some people say oh well
then they're not really interested in my
giving feedback itself absolutely that's
exactly what you want you want somebody
objective who doesn't care around your
project right they need to just be it's
them establishes the terms of a
relationship right this is a trial
action between your your sharing your
knowledge and I'm asking you questions
in addition paying them means they will
probably turn up this is the biggest
pain in the ass for a researcher for
your research participants to not turn
up you can't just magic another one out
of the air for that power slot you've
got your observers lined up you know you
made booked a meeting room somewhere as
well so is well worth it yeah five
dollars
no is absolutely not enough money then
that's just the failure that's just the
reality of some people right you can't
necessarily access every single person
so understanding what the incentive is
that was going to attract that person is
really important I've done research with
like CEOs of major big or corporations
they don't want money they want a
charity donation in their name
to think about what the alternatives are
be aware if you use bouncies then you're
only going to get users likely to the
Bounty's platform we already going to
get people who use the right so it's
quite small and its really annoying
because so when I don't be to be
interviews I sort of a phrase it like I
want you meet your expert opinion right
but people love being heard and they
want to be considered the you know the
industry somehow have to react to their
world they're you know pearls of wisdom
right but but it burps but yeah it
shouldn't be a two-way thing right it
needs to be I'm researching you it's not
useful to you if there does not come out
with those questions for you so it's
about establishing that the rules of
engagement right this is a research
session I'm going to be asking new
questions about this I pay people like
five hundred dollars for an hour at
their time depending on who they are
right and you can find you
you can find people in specific roles
and specific companies through these
platforms as well so it depends on the
other thing as well there's always going
to be a bit of bias as to why people are
motivated to take part that's that's
just reality that's just real life you
know but speaking to a few of them is
going to raise the confidence and
responses there any other questions
about recruiting or paying people okay
just be aware of you do insist on pay
people have done or easy you're only
gonna get people who are happy with that
right so just consider that also I I
don't do that because I just I live just
next level I couldn't get it financed to
be able to work out how to receipt it
properly so just like you never seen it
announcing it coming
why don't you hate the company or not
how many people so you probably heard
five is the magic number for users right
there's actually some science behind it
and it's it but it's quoted incorrectly
I'm not going to go to loads of detail
about it now five it's it's like how
long it's a piece of string five is a
good number to start with per user time
right it's not just five one want that
developer one crypto trader one asset
manager you need to do five of each of
those right to understand the themes and
patterns okay but you really do it
enough that you stop learning right no
users no insights one user loads of
insights two users not twice the
insights because there's probably going
to be some overlap between them so
you're going to get diminishing returns
and I'm working on a study at the minute
about I defy users people using maker
and compound and things like that and
we've got two about twelve and now I'm
not learning anything new right so
that's really a good indicator that I
can stop now formulate these insights
we make the recommendations to the
business and it's about doing this
little and often so if you imagine that
your company is like a ship right you've
got your end pointers in sight or on the
back right but you need this constant
course correction it's much much better
to do a few users often than it is like
20 users studied at the start and
they're never doing it again right
because you'll probably be well off
course by the time that you get to
launch your product so just doing a
little bit often helps correct and gives
the people purpose and alignments around
what they should be building - any
questions on numbers cool okay so the
set up alright we've just aimed in
person if you do it in person that's
great there's pros and cons for both of
these things so I do most of my by video
call the reason is that like we're
distributed company I work from my home
and sometimes I get like Sasha such a
world dial in with a camera off you know
and sound off and she'll be the observer
right and I'll explain to the user I've
got my colleague Sasha on the call she's
just taking notes for me and so it's not
a distraction having like loads of faces
popping up right summation of videos on
yes so why would you have those second
first anything at all it's already
recording it I challenge you to go
rewatch your videos it's very time
consuming it most people don't watch
videos because it doubles my time so I
do the interview and then I would have
to watch the interview again
so if you do a synchronous it's also and
people don't watch but people just don't
watch the videos I know we said some
people might some people might in my
experience I think you know the CEOs
weren't really interested on how to do
it send them links they don't they don't
watch the videos right so the video is
there as an artifact for me to rely on
in case where you've it up right
in case I can go back to them what the
hell were they saying when I my notes
don't make any sense in this part I can
go back to that you might make like a
highlights real for people if you really
want to do that that's quite that's very
time consuming but also very powerful as
well
willing to behave differently when he
knows that his life never that doesn't
record absolute they pay attention right
so when you're watching your video but
you only ever just watching a video
you're on slack your email texting
whatever right The Observer has to pay
attention exactly when I get there and
there's some training that there are
some positives to doing video calls as
well because it requires less less of
the user to take part they can do it in
a time and place that since then they're
booking with you and it's it's easier
for them to take part if you require
them to be in a location you have to pay
that loads more money to turn up and
also there are some problems with
inviting them to your office so let's
say you work it's really cool tech
company godlike it's alright really cool
post-it notes everywhere and like cool
furniture and stuff that can all
influence the users perspective of your
ground and your company I've used like
like look like meeting rooms by the hour
and like anonymous sort of places I've
done that before
meet them in like a coffee shop I guess
as long as it's quiet enough for you to
have a conversation without it being too
disruptive that could work as well but
vehicles they work and we work we've got
global audiences the old customers all
right so the days of the usability lab
of well over right that's old-school
approach to doing testing and interviews
we don't need those anymore there's
better ways to do this that's much much
cheaper and it's doing this well there's
a difference though between you doing
usability testing a very structured
sequence and you're also eye tracking of
you're seeing where well button they're
looking at before they click that would
be the old-school usability microscope
compared to this is more subjective
fluid is that correct yes what we
focusing more on that on the dialogue
element but I have you know my tracking
looks cool but I that tells you what not
why right
so I tracking says other user look to
that and they look to that limited than
that but you need to ask them questions
to listen to why behind it so you still
need all that other stuff
I personally not a fan of some of those
things as if you're marketing of course
at UCL in London around this where
there's like movements into you know by
putting my by putting people through MRI
scanners or stuff to try and understand
their brains and their behaviors and
it's still guess it's guesswork at this
stage it's just you know went to the
course and say well I can't to be honest
just interview
get the sacred sites service yeah
but no we there was a place for
usability studies because there was an
easier technology to do it so is it
trade-off right you might get the most
in-depth usability report possible in a
lab right but it's that worth in it than
just doing you know a few users often
there's a trade-off for things to
balance this one question is you know
I've been both worth it user experience
design management and so on and I've
also been a stealth subject for a big
software company as I was a I was the
guinea pig but didn't abuse those people
with the whitecoats and so on yes scary
though I went through a sequence and I
did my own thing and then you know said
goodbye their product was for yes this
is that over reliance on technique and
science and software I think we put a
tracker on they will know what's going
on in their brains it's not necessarily
true you know it's just having able to
have the be able to ask these questions
as well yeah
things like watch are there maybe
perhaps well so kind of things firstly
the hot jar you need users so you mean
as it known as it not like it gets a
test you actually need a large song poor
people to be doing it so one of the
problems with that is they've you're
early in the crypto space you may not
have many users in which case you just
not gonna get a lot of insights out of
it basically the same problem as Google
Analytics using about your platform is
watching one session which is extremely
useful so you have like basically one
year ago to a global command and seeing
how they do things and if you watch like
them like when you session to get a
better feeling of how people are yeah
yes so the real benefit of that is that
those are real people doing real things
at your site it's not a made-up user
test right that is absolutely beneficial
the only thing I would say is it needs
supplementing because it's still up or
what it's still one more else are they
doing normal I yeah but absolutely I
agree that the question is you know
because you don't understand their
context you don't know what else is
going on in their life and I've watched
some of those videos and then then
they're not doing anything for a while
what are they doing they got a toilet
like you know what's going on in that
time I don't know why they make pauses
when they do why they click on things
that they do what they're looking for
when they start but they can be
interesting I think this is like that's
what they write and or elements of it
sort of bridges that it's like you can
dive into that data to seal the room you
know level but also
the collection I think all successful
companies these are both of those things
that you could you can make your
assumptions based on information right
he's like Oh
everyone goes to this page and then they
do this thing that's weird I wonder why
they do that and then go find out yeah
exactly yeah so I did this proof above a
shop so the last thing is about couple
more things is video conferencing right
so we zoom but the amount of target
service generally easy but the amount of
times that I had a session where the end
user has not been able to use the
software that you demanded that they use
right is like it's it's just after that
what's easiest for them and you do that
right so the easier for you to set it up
at your own to make sure the the you
know spend the first half of the
interview helping the work out where the
share button is right so this is quite a
good tip it's just to to do go where
they are in terms of the software that's
all use record it but do always ask
permission beforehand so this comes to
that is really beneficial to record it
so you can come back right don't just
ask them at the start of the session but
by the way I'm recording in ask them
like when you're recruiting them okay
because then they'll always say yes if
you land it on them last minutes then
people don't have being surprised about
that kind of thing at so it's a it's a
good idea not to just yeah like it on
them like that you might have consent
forms or nba's that's down to you as a
company I mean so I think some companies
really overhype their own the
the information that they think they're
going to be sharing with this the user
and rightly you're asking the user all
into questions hopefully you're probably
not going to tell them loads of stuff
that you're doing if it's secret right
so just like it's a maybe bit
heavy-handed in some context some
companies insist on it speak to your
legal team if that's the case most of
the projects I work on we don't use
FDA's so the last bit we're nearly done
I think yep so it's just about making
sense of the findings right so you've
got all these patterns of things now
that are coming out you're going to
create recommendations and actions so
you go to come up with I think we should
be doing this that's and it's good to do
this as a team as well but what you need
to do is prioritize them because you're
like especially if this is your first
research project you know it's like
low-hanging fruit you're gonna find our
loads and loads and loads and loads of
stuff you want to do or change about the
product in which case is worth plotting
it on something like this
so this x-axis is like technical
difficulties and up there use the value
or remedy so what you can do with your
recommendations is as a team decide
where what quadrants the recommendation
goes right so if it's low technical
difficulty but really high using our
annual revenue it's in the top left easy
and important those are may be the
things you should work on first stuff
there's high technical difficulty and
high value important that's big so maybe
work on that in the background and
you've got stuff that's low value and
low tech
difficulty not important to be working
on like wait there and then you can go
to the CEO who's been demanding this
feature here in the high technical
difficulty learn user value and say look
we've plotted it's it's not worth doing
right so let's focus on the top of their
things so this is just it's really
simple activity you can do in like an
hour from the research that you've done
and it starts getting all the T might go
alright that's the stuff we're gonna
focus on you can even do it in that
session who's gonna work on what create
your tickets for it right further for
whatever you're doing so it's it's not
rocket science it's just getting to this
point and remember that it's not it's
not certainty it's always about course
correction okay you you're gonna have
new assumptions and new hypotheses that
come out of the research that you just
done oh god we only thought that's no
way to explore this to research usually
leads to more research but so so yes so
we've got like 10 minutes left the last
things just want to do is just give you
some what we recommend as great books so
the first one Daniel Kahneman's Thinking
Fast and Slow it's all about cognitive
behavioral science the cognitive biases
that we have of why people do the things
that they do if you can if you can read
this and you'll hopefully have a
slightly better understanding of how
people respond to your product use a
person and how you respond to the world
the book in the middle is an excellent
resource for learning to do UX research
especially if you don't have a
researcher and it's great it's easy to
read it's quite entertaining and I
recommend it and then speed or particles
interview users is more detailed about
uncovering insights through into young
people or techniques and the very last
thing is we need participants consensus
research that we like that you can't
hide so if you fit any of the following
criteria that's all please email design
research at consensus don't mess or scan
the QR code I'll take you to it it's
pretty fun that we got out of our
bubbles okay I guess this is sort of a
bubble and Epcot but this is the most
bubbling it gets but you know this isn't
specifically looking for people in like
this region or who are defy users or
adapt developers you don't have to be
all three of these things right just any
of them drop us an email and that would
be really good so I think we could maybe
just take some questions in the last
couple of minutes if anyone has a name
are you going to share this slide this
one the whole tag and we can do but I
don't know how what the method is for
that sure if you want yeah everyone's
copy than yet so again based on your
experience and well I think what can you
work
and in your experience was different
research methods you can you see
specifically for work free if you see a
different surgeon to see a mere human
see I can go first and then you yeah so
my personally no I think the attitude
the attitude the attitude problem
thought the teams is different but the
techniques are the same the youth the
people might do the same as well and we
don't need to reinvent everything that
we do right there are some try to tested
things that work or technical comfort or
big tech companies doing this so I what
my experience is and that's why we're
doing this workshop here is we've seen
lots of crypto project teams just not
not be down to try and doing this
because it's not in their wheelhouse
they haven't had experience doing it
before and they're on the assumption
that it's blockchain is so growing you
no one knows what they need so there's
no point in asking them right and that
comes to that this problem what I know
teams will be building things for two
years and haven't found a use case for
it right
what a waste of money and effort time
[Music]
I mean that's what I get don't you think
it is it is still viable to conduct user
research in each stage of the product
life cycle
yes absolutely so we're encourage you to
do it early but it's like I guess better
late than never right and and also we
you know enough promotes of products
with like really big you says that
turning over 200 million yeah but we do
testing all the time one big challenge
you need to there's a lot of friction
with project managers with developers
with all those type of people what is
the best way you could confess the team
that we really need to do that user
search how to is yeah it's about you
starting the first time is difficult I
yeah because people are like we've
talked about the biases people are
resistant to hearing feedback or they
they're not quite sure how it works and
what they're going to get out of it
my advice is definitely to include them
don't just do it on the side because
that's going to make you look like you
don't trust any of their opinions right
and then you're just going off and do
some secret research and then you're
like wow I suppose these users like
you so that's not to be an approach this
book the one in the middle addresses
this exactly right it's got like that
you can get this free or - it's like
right but by a book and he's it's also
forgetting buying
absolutely the most compelling
experiences that I've seen are insisting
that the person who has the scoop is
resistant to research observes all the
sessions right get them don't rely on
them to watch any video they won't watch
the videos I get your the observer comes
in they're here and once they start
hearing from people they can't button
and not allowed to say anything during
the session they're just really like
okay Rapids do more of this right but
it's getting to that point which is
tricky we're hoping that by you know
what what in consensus design there's
quite a few researchers we're trying to
do this with all companies so I like to
I do like a little bit of like I'm going
to talk to or spoken to gnosis before
about doing research to help them
understand like some of the benefits of
doing other they're already doing in so
that's good so it's just about education
in being district and I think a lot of
sort of trouble I think a lot of teams
are you know lots of people made lots of
money and I SEOs and it's got nothing to
do with their ability to run a company
and as a result they're necessarily are
the right questions so I'd like to add
to that that you know companies were
first of all division is not clear that
the strategy like they're the main
objective or
the main key technical some of those
they're just ready how to get there and
then if you go further further than like
use cases and also like non-existent
yeah building very cool yeah it's really
cool but cool is also do discomforts a
lava and feeling else has to do a bit
with the maturity of like a maturity
level exactly yeah really makes sense to
me because mothers are a lot of young
people yeah so we're gonna I'm gonna
waited to create a consensus Academy
like the modules I'm doing user research
up and trying to get out in front of
people so so they can just try and you
know these skills and sales you know the
UNIX researcher your job was to educate
doctors and educate your team and then
why is so important so I just wanted to
dub it down to more course to do you say
if you have a resistant or that existing
team member you can put him to start as
an observer yeah
so that's observer is like the most
comfortable position among the three
exactly and they don't have to do
anything like take notes and then just
listen I take that it's yeah so it's
true twice I'm no taking is that when it
down right everything yeah and I just
write it all down every single thing
that said because I'm you can interpret
it later
yeah
just take her all day after like after
position you didn't observe or do they
do they do they like ease by beginning
they they are very resistant but after
being observers they change my advice is
to get them to listen to - or home
because one of the things that they can
do is just latch on to that one user
that they permit of God right we need to
change everything right because it likes
a really good session and my answer was
ready for both but of course that's not
use research used to be to a few people
so just like more than one very original
bias they might hear do you remember
that user he said it's okay no I mean
you know at least I want to listen to
use this as a star right even if they're
hearing what they want to hear it's a
really good start to get them interested
in use they're doing the sessions - okay
any other questions for me
rather particular what he comes to need
but learning for me is where I should
not really hurt people in this room even
though I love you to think we are what's
here what works for you guys in terms of
like to finding you people around
thieves like general consumers I'd be
morally in favor of you know using
products that 2x might say as opposed to
others more like recruiting people
around things really yeah so ignore
demographics demographics are not really
overrated their proxies for for that
what you're actually interested in who
cares how old that person is right and
you know how much the household income
is if they are behaving in the way that
you're trying to find right so within my
platform like this or with any
recruitment use a screener a screen and
a series of questions to determine
whether they are the right person for
you so for example I was finding defying
users this is an example I mean people
who use make and down stuff like that I
ask them first of all do you do you have
cryptocurrency yes or no like like a
kind of starting for award getting more
specific just this as you doing
interview itself and then like what
coins do you hold allow them to like
type them all in so so you can tell
whether that sounds like they do know
what they're talking about or not and
they like which of these things if you
use before a long list of all kinds of
different apps and there are the defy
products in there and they have to have
selected one of those in order to
qualify for the research so that's the
behavior behavior is how they've done
this thing in the last six months that's
what I care about
I don't care how old they are you know a
lot city they live in that's not really
very useful information again
anything else well one thing is in issue
is you know I think user research is
great primary source understanding you
know there's a separate function lots of
companies is totally silo to break and
that's really frustrating
I had to build relationships with the
data analytics teams and other company
that worked out probably you're probably
on a smallish team that it's not such a
problem to be able to reach out to that
person but we should be working together
don't really see why why that hasn't
been the case probably if you work
encrypter or depends on what your
product is if it's a private product
after launch you probably don't have our
analytics of
but I kind of a say they need to be
working together you need to perform
your assumptions your hypothesis from
that you know from what you're seeing
where people what countries people
visiting for are mapped you know where
they click and how often they dwell time
of things like that or things like hot
you are as well and they can help form
your hypothesis but it won't tell you
what the intrinsic leaned is of the user
which is the thing you can improve on
Center you can build onto a prenup right
okay
yeah just drop myself just drops me you
know you're interested in buying
research come and talk to me afterwards
if you like I may be around for an hour
so I gotta fly to avoid my foods
[Applause]