[Music]
hello everyone
hi thanks everyone for coming I'm uh
really excited that we've got a sizable
crew even though this classroom is
pretty much impossible to find I don't
know whether anyone else struggled to
find it
yeah so congratulations you passed the
first test you can find uh a classroom
in the middle of nowhere um so as you
can see this will be all about doing
user research or ux research in web 3
we're kind of going to use those terms
interchangeably and uh I'm not doing
this myself there's another person you
wouldn't know because she's hiding right
behind me uh and we're going to walk you
through this beautiful journey of doing
user
research QR
QR there a code yes so um because it's a
workshop you you don't have to only ask
your questions um using the the mircat
app but feel free to to do that like we
are going to check the app and um at the
end when we're doing Q&A if there are
any questions there we will answer them
um yeah so let's get
started nope yes these are our names
hello um just to give you an idea of
what this is going to all entail we're
going to start with intro and it's
already happening so look at us doing
great the first kind of half of this
Workshop is going to be a crash course
in ux research so what it is how to do
it what tools to use what not to do
that's also an important part of it and
then the second half will be an exercise
when you can apply all these um facts
that that you learned today and then
we'll have some time for
Q&A all right so um let us tell you a
little bit about ourselves uh Mindy
please emerge from your cave and tell us
a little bit about
yourself hi I'm also here my name is
Mindy um I've been working in ux since
uh 2017 working in web 3 for about 3
years now uh I am the senior ux
contributor at cow we are uh building
several products including cow swap
which you may have heard of um didn't
get any any Applause that's sad
um thank
[Music]
you cow team are you asleep what's going
on uh yeah and then my um my lovely um
assistant here not assistant wow
co-host yeah so my name is Christina my
pronouns are they them and I've been
working in crypto since uh 2018 most of
the time has been osis and then safe
this is where we hear thunderous
Applause for those
companies um mostly as a as a ux
researcher and uh I'm based in Berlin as
you can tell from how I
look all right so now you know a little
bit about us but we want to know about
you so yeah uh raise your hand if uh you
feel like you are a complete beginner in
terms of understanding ux user
experience Ono put your hand
down okay very few people okay that's
really nice raise your hand if you feel
like you have like a pretty good
understanding of user experience user
research what was it can back to
question one if you're a
beginner oh okay okay back to question
one raise your hand if you feel like
you're a beginner in
ux okay a few more people have decided
they feel like beginners raise your hand
if you feel like you you understand ux
pretty
well okay all right pretty good raise
your hand if you uh have already done
user research on a web 3
product Oh okay nice yes cow guys come
on okay good so yeah so the one's
already doing research great it's good
to see you here as well um this uh talk
is going to mostly be um sort of basic
um logistical information how to get
started doing user research but I think
there will be useful tips for people who
are at any level um of doing user
research or learning about user
research all right so um let's get down
to business what is ux research how does
one do it all that good stuff
of so ux research is all about
understanding your users or your
potential users in order to make better
product decisions that's that's the
purpose of ux
research there are many many ways you
can do it and there are kind of two main
types there's quantitative and there's
qualitative for quantitative you usually
need more users for it to be
statistically significant 40 is about
how many you need for example for a
survey to be valid and this can tell you
how many people are having a certain
problem how much of a problem it is so
if you want to quantify certain facts
that you already know a little bit about
that's that's a good a good type of
method to choose there are many ways you
can do quantitative research there are
surveys card sorting tree testing
benchmarking if you have no idea what
these words mean don't worry
um we're not going to talk about those
today spoiler aler then there's the
qualitative side so there you can get
lots of information from just talking to
a few users so five people already
provide lots and lots of useful
information for you and the kind of
questions you might get answers for in
terms of qualitative research is why is
this happening why are you are users
behaving a certain way how do we fix
that is this solution suitable how do
people feel about it and common methods
for qualitative research include user
interviews usability testing diary
studies field studies again this is not
an exhaustive list at all uh people have
been really creative in terms of coming
up with different ways to conduct
research so
as you can see this is an impressive
list because we have two hours we're
going to do these two this is what we're
focusing on today and the good news is
these are quite easy to learn and you
can get a lot of value from from doing
this kind of research um without
spending too much time on
it so let's start with user interviews
what they are and how to do
those yes so uh user interviews I like
to summarize user interviews as um the
art of caring intensely for the space of
like 30 minutes to an hour uh about how
a total stranger thinks and feels so um
the way I will um kind of illustrate
this is you know how that that feeling
that you have when you're falling in
love with someone or you know you really
infatuated someone and you just met them
and you just want to know everything
about them and you just like they you
just want to hear everything that they
what they that they think and like every
sentence out of their mouths is so
interesting to you that's how you need
to feel about your user just for the
space of an hour afterwards you don't
have to care about them anymore but for
the space of that hour you have to care
so intensely about what they're saying
that's the mindset that you need to be
in in order to do a user interview
effectively and um what I mean by
that uh is is that user interviews
basically are a way of uncovering your
user needs um uncovering their pain
points so that you can optimize your
product to fit them so the goal is to
learn as much as possible about this
person's thoughts fears wishes needs
habits um as related to the area the
topic of study uh so that you can build
a product that will make their lives
easier and make them want to use it what
do you need to do a user interview uh is
basically a list of questions you you
need a participant obviously um ideally
this would be somebody who is a user of
your product or somebody who is a
potential user of your product um and
then you need a way to take notes or to
record the
conversation uh what can we learn from a
user interview specifically so from an
interview we'll learn what this person's
uh needs and wants are uh what's
important to them what people aren't
getting from existing Solutions uh that
are out there which could potentially be
advantages for you uh and then also how
people make decisions about which
products they use um and the tradeoffs
associated with different using
different products in web
3 um now if you sit down to write a list
of interview questions uh for a user of
your product you might be tempted
immediately to ask things like will you
use this product uh do you like this
product isn't my idea really great um
and those are all terrible questions
don't ask them ever
um the goal is to get people to open up
and share anecdotes from their lives
this is not like um like a Q&A interview
um or you know like a I don't know a
survey this is really you want them to
be talking more than you're talking um
you need to get them engaged in the
conversation thinking deeply about their
answers and telling you oh yeah and then
there was this one time where this
happened and that's happened that's when
you get really interesting moments from
their lives where you're like okay they
actually really had a problem here and I
think maybe we can solve this problem um
so here are some guidelines whoops here
are some guidelines um for writing these
kinds of questions uh the first one is
to ask specific questions general
questions get you general answers and
you don't need general answers uh when
you're building a product you might end
up with something like okay so I spoke
to 10 users and they all told me that
they use exchanges a lot great how does
that this help me build my product uh
compare that to something more specific
like I spoke to 10 people and eight of
them used an exchange in the last month
to swap meme coins okay that gives you
more information about what people are
actually doing and then ideally you get
the why about that as well and then
we're getting into the really
interesting specific data that can help
you uh Market your product or build your
product to to suit a particular audience
um you also want to ask open-ended
questions as much as possible not yes or
no questions at the end of the interview
you don't want a list of uh checkboxes
yes no yes no uh again you want those
really detailed stories from their lives
um so you really want to ask open-ended
questions so then that the participant
can take that question and and and start
answering and then and then snowball
from there and take the the topic to
wherever they want to take the topic to
wherever is really interesting for them
or where they really have a product to
vent project a problem to vent about um
another one is don't ask leading
questions um so people will pick up if
you have some bias uh everybody has bias
obviously we try to keep that as low as
possible uh when we're doing research
but if you ask bias questions like um
crypto is pretty complicated huh then
you already have put that idea in their
heads maybe they weren't actually
thinking that crypto was that
complicated um or maybe they maybe they
they they want to you know go along with
what you're saying they want you to feel
comfortable you know they don't want to
stand up against what you're saying so
they'll just go along with that um so
you instead want to ask a very neutral
question like you know maybe how would
you rate the complexity of crypto and
then see what they say about
that uh another is that you want to ask
about past experiences not future
predictions uh in Psychology we know
that people are pretty bad at predicting
their future Behavior Uh how many of you
have ever sworn that you were going to
start going to the gym regularly and
then not done
it
yeah
yeah okay everybody here goes to the gym
regularly another thing about user
interiew that sometimes people lie yeah
yeah as we see
here uh the point is that people are
really bad at predicting what they're
going to do in the future they might
have intentions to do something but it
doesn't mean they're going to do it and
they might do things that they had no
intention of doing um so it's always
better to ask uh about what people have
done in the past rather than asking them
to predict their own future which nobody
can really do um and then the last
guideline here is to always ask
follow-up questions and this is where we
really get into that part of being in
love with the user because you know they
say something and we don't just say okay
yep that's all I needed thanks we
continue okay why did you do it that way
oh my gosh that's so interesting really
would you do it again the same way the
next time or or would you do something
differently if you could wave a magic
wand and like make it different you know
make something different about this
experience what would you how would you
do that can you imagine a better
experience can you imagine a worse
experience can you compare it with other
experiences if you had any of these
these follow any follow-up questions
that comes to mind is is useful and it
helps make that person feel like what
they're saying is important and it makes
them want to talk more uh and it gets
you really really uh indepth uh
qualitative data that you can then uh
use in in your product
building and uh
Christina smooth
transition um so that was user
interviews now I'm going to talk about
usability testing so there are some
similarities you are still talking to a
person but if you think about what you
just heard about user interviews you you
really really care deeply about this
person you want to know everything you
ask them all kinds of questions you're
really engaged you're really present
there usability testing is a slightly
different vibe so with usability testing
you're watching someone fail and you
just got to pretend like you don't care
at all you do care like let's let's be
clear in order to do good ux research
you need to have empathy you need to
really care about how people feel what
their struggles are that's all true but
on your face ain't nothing
happening so try to cultivate some kind
of inner
Serenity so sble someone stumbles and
you don't help them up they ask for
directions and you don't any it's kind
of a struggle at first for some of us
but that's really how you get the most
valuable insights we'll get into details
about why in in a
moment but first let's talk about what
you need to do usability testing so
unlike user interviews that you can just
do you know sitting in a cafe with
someone for usability testing you need a
little bit more of a setup so you need
either a live product or a working
prototype when I say working prototype I
don't mean it's you know implemented in
any way it can be just a series of
clickable screens in figma or something
like that that's totally fine you also
need a list of test scenarios we'll get
into that in a moment again a
participant it's kind of sad to be just
sitting on your own uh trying to imagine
things um side note AI is not a
participant you need a human
participant so um same as with the user
interviews it needs to be either someone
who's already your user or is similar
enough to someone who your user might be
and you need a way to take notes or
ideally record the screen as the
participant uses the product because
that's how you're going to really see
what they're doing see what they're
clicking on see their thought process
through the way that their cursor moves
that's all super super useful stuff you
don't want to miss it and that way you
kind of will be able to uncover those
problems with your design so that you
can improve them in terms of clarity and
flow and ease of
use so what can you learn from usability
testing you can see what it is about
your product or someone else's product
can be a competitor that makes people
struggle or get confused or make
mistakes so this is where this stumbling
thing comes in that's actually good like
you want to watch people struggle
because if someone does everything in 3
minutes and has absolutely no issues I
guess it's good because you found the
product is easy to use but you don't get
any insights in terms of what you can
improve and your goal is to come away
from the test knowing okay this screen
is confusing here the button is hard to
find um the text makes people think that
the feature is not what it actually is
all of those things
so it kind of counter counterintuitively
you want people to
struggle um but you can also learn what
people like about the product they might
just say oh wow like this this really
makes a lot of sense I was waiting for
this feature I've I've heard that many
times it always is super nice to see
people excited about something that
you're building um and also you can get
ideas about what else to build so how to
build maybe the V2 of the of the feature
that you're thinking about um so all of
these you can
uncover so what do you need to have in
order to do usability testing you need
to get your participant to navigate the
UI and if you're doing it online which I
think is how we mostly would do
it hello hello yes right um let's
maybe exchange this one thank
you
perfect
um where was I yes so if you're doing
the test remotely have the tester share
their screen so that you can see what
they're doing um because that's that's
where you get all those valuable
insights and ask the participant to
accomplish some some task maybe task is
not even the right word it's more like a
scenario more on that very
soon again it's important for you not to
interfere as much as you possibly can do
not nudge the tester to a certain
correct path do not guide them as much
as you can if you see someone really
really struggling you can you can ask
but wait just a little a little bit
longer even though you you're like oh
this is so obvious you really you if you
just like clicked on this button you
would get everything you need don't do
it don't do it observe what the user
does because that's how you uncover what
your users are likely to do in the real
world and see how they solve the problem
and that will help you understand how to
improve so on unlike user interviews
where you can just kind of go right
ahead and start interviewing the person
before you do usability testing you need
to prepare the person for how that's
going to work because it's slightly
unusual for someone to just sit in front
of you or even in front of their
computer and do this kind of fake is
scenario and and pretend that they're
accomplishing something so you need to
get them in the right
mindset um tell them to just think out
loud and be completely
unfiltered if they like something if
they don't like something they should
just tell you everything and tell them
to even pretend like you're not there
not in the room or not on the other end
of the zoom call just your goal is to
have them act as naturally as possible
it's hard and there's always going to be
an element
of uh
awareness of being observed but as much
as you can encourage them not to do that
you should do it and remind them that
you're testing the UI so the Prototype
the product whatever it is and not them
this is very important because people
get self-conscious people start worrying
that they're making mistakes people are
starting to think oh no I messed this up
this person is going to think that I'm
stupid that I don't understand things
but I'm so smart I've I've been working
in this industry forever how do I not
get this people get it in their head
that's normal that's why it's important
to remind them this is not about you
this is about the the UI uh if there if
something about it sucks we'll fix it
don't worry and also you you can even
tell them I I didn't even design this so
you're not going to hurt my feelings if
you hate it even if you did Design This
say that you
didn't because that just puts people at
e about criticizing something not
everyone is comfortable with that and uh
if it's a prototype make sure to mention
that it's a prototype because some of
them are pretty highdea and people
assume that they can just click on
everything most of the time that's not
the case make sure people know
that okay so now we get to the tasks
things so as I alluded to don't think of
them as tasks because tasks are kind of
efficient and that's not your goal your
goal is to get someone to behave
naturally and people aren't always very
efficient the way they do things
naturally so how to write out a test
scenario choose a flow or a feature that
you want to test for example it's a
really really important flow for for
your product so if you're just starting
with with testing can be I don't know
for a wallet sending money checking
balances connecting to a dab stuff like
that um it can be a flow that you know
are is a particularly tricky one or
really really risky one it can be a flow
or a feature that is new so once you
kind of get into the rhythm of doing ux
research um when there's a new feature
coming up that's a great a great moment
to to do um the testing because that way
you uncover problems before the feature
is implemented
and you change the design in advance so
you save
time or you can test changes to up to
existing flows that you know users
already comfortable about but it might
be different so you want to make sure
that it stays comfortable for
them so as I mentioned scenarios are not
tasks so don't say okay so now click on
the sign up button and fill out the
fields with your information that's not
a scenario that's an
instruction and the goal of a scenario
is to give the Tester the minimum
absolute minimum of the required
information and learn how they approach
the the the scenario how they would do
it given their own understanding their
own assumption their own habits
so for example if you're testing an
exchange you can say okay imagine it's
often good to start the test scenario
with imagine imagine your favorite token
just dropped 20% you want to take
advantage of this opportunity to buy
more how would you do that so you see
it's it's kind of phrased more like a
question like a hypothetical scenario
and that invites your tester to do
whatever it is most natural for them to
do and then you learn about how they
would do it in your actual product or if
it's a wallet you can say hey imagine
you made a new friend at Devcon stayed
out all night talking about ZK proofs
now you need to pay him back for for
dinner how would you do that and then
you sit back and
observe there will be struggles in
usability testing that's kind of the
name of the game as I've been telling
you and
there are different ways to handle those
struggles so for example the participant
can ask you a question but not just like
hey what's your favorite color um it's
like how do I do this or what do you
want me to to click on or how where do
you want me to navigate
to or if I do XYZ what will happen this
is the moment for you to tap into that
inner Serenity and be like that's a
great question I'll answer it at the end
but for now just do whatever feels right
I want to learn about your assumptions
your habits what you would do if you
have any questions at the very end of
the test I'll answer all of them if the
participants seems
confused which most likely they will be
um you can kind of nudge them to explain
a little bit more about what is causing
the confusion so can ask what do you
think is happening like I I see I see
that
you're wondering about something I see
that something is giving you
pause what do you think is the issue how
would you solve this problem because
sometimes especially if it's a prototype
you you can't really do everything and
people normally would I don't know close
the app open Stock Exchange ask a
question there or like Google something
so it's good to also give them space to
to tell you about
that and so if the participant is really
really really struggling so they're
about to lose all of their
money um rage quit the the test and you
can see that they're just not getting
anywhere you can say all right so I
think I I got the information that I
need would you like some help don't say
the thing that needs to happen right
away unless again like they're about to
click on the button that is going to
destroy their life forever
um unless that's happening just ask
would you like some help and
then let let the person tell you or you
can just say okay let's let's pause with
this one and go to a different flow do
something
else so
this was the O overview of usability
testing um but these principles apply
to just doing ux generally so studies
show this not us saying this people have
looked into this and the study shows
that for usability testing just five
participants is enough to uncover about
85% of the
issues as long as those participants go
through the same flow so if you're
giving people the app and saying oh can
you use it for a bit and and do
something um then no five people aren't
aren't enough but as we said before you
need to be specific with your scenarios
with your questions so as long as you're
specific and and you give the same
scenario to everyone five participants
is enough to uncover a lot of the
issues that being said you need to make
sure that you're testing with the right
people if you go out into the world and
your product is geared towards protocol
Developers for example and you get five
casual coinbase users that like have
heard about
metamask your insights will not be
relevant to what you're building you
need to make sure that you test with the
right
users and finally we love ux research
like that's why we're here but it can't
answer every single question that you
will ever have it will tell you about
user needs habits motivations pain
points that's a lot but it cannot tell
you for sure whether your product will
be successful or whether a particular
feature will be actively used by
everyone that's just there are some
limitations to what you can accomplish
with ux
research as you're uh as you're going
into uh starting your your your first
research um studies or continuing the
research studies maybe that you're
already doing um it's important to keep
in mind especially in web 3 that web 3
users are very privacy conscious um as
they should be um they're often very
suspicious especially of things that
sound too good to be true of offers for
money in exchange for doing something of
random people sending them messages on
random platforms again as we all should
be um so it's very very important it's
important in any area of uh of research
even if you were doing e-commerce or
something like that but it's especially
important in web 3 that you give users
as much information as you can upfront
that you're very upfront about what it
is you plan to do what it is you plan to
ask of them what it is you plan to do
with their data um and that you keep the
requirements to join your study at an
absolute minimum um so some examples uh
the first thing you we want to do is you
want to inform somebody when you're when
you are approaching them or when you're
putting out a call for for um to recruit
participants um you want to inform
people what's going to be involved are
you going to require them to show their
their faces on video are you going to
require them to speak um what language
are you going to require them to speak
and do they have to speak perfect
English um do they have to share their
real name do they have to share their
real wallet address do they have to
share their screen do they have to
connect their wallet to adap uh do they
have to do a transaction with their
wallet do they have to uh will you take
a recording how will you where you will
store that recording um all of these
things uh are things that people need to
be informed about upfront so they can
make that informed decision about
whether it's worth it to join your uh
your study and so then it's best for you
if you have kept those requirements at a
minimum so that then people feel like
okay I've been told that if I join this
study I don't have to connect my wallet
to any dap and so if I believe that then
there's very low risk for me right to
join this study okay and then you you're
more likely to get participants if you
expect people to you know just hop on a
Discord call with you and share their
real wallet where they have all their
money and uh you know connect to your
your dap that's probably a hard sale um
also make it available that people can
ask uh to request deletion uh of their
data that you've taken from them of the
notes that you've taken from their study
after a certain point um let users give
consent granularly so ask them not just
yeah yeah these are my requirements do
you consent to these uh what I typically
do is say hey I would like to take uh I
would like you to share your screen I
would like to take a recording of that
screen and of your audio um but you can
consent to these separately so you can
consent and say yeah I'll join your
study and you can watch my screen but
you can't record it let people consent
granularly to each different thing that
they're willing to share um and that
also makes them feel a lot more
comfortable um joining your your your
session um and again keep it Loose if
possible if somebody says no I really
don't want you to take notes or I mean I
really don't want you to record um then
just kind of be willing to say if if if
possible we be willing to say like yep
okay you know what that's fine I'll take
notes by hand if you're willing to kind
of slow down and speak slowly so I can
take those notes and people will be um
understanding of that um another is that
uh if people are going to be doing a
transaction for for this test um you
need to provide them with gas if they're
going to be swapping funds for this test
you need to provide them with the funds
for swapping never expect people to use
their own funds to join this uh this
this test of your product um that's not
ethical and um you know it's just also
kind of a bad sale It's A Hard Sell so
yeah always make sure that if you're
providing funds also provide gas funds
as well if they're necessary um if
they're sharing their screen uh let them
know that they only have to share um the
tab that your uh that you're d is in so
a lot of people will just automatically
share their entire screen which means if
they switch over to another tab if they
open another app if they open their
metamask you can see all that you can
see what's in their metamask they
probably don't want to share that with
you so guide them to only share the tab
that is relevant for you um and then uh
yeah keep info sharing requirements uh
to a minimum tell them they don't have
to share their name tell them that they
can refuse to answer a question uh if
they don't want to answer that question
uh phrase questions in ways like would
you be comfortable telling me or would
you mind sharing you know how many the
size of of a transaction how many do
dollars you trade in a typical
transaction some people don't want to
share that some people are fine with it
it's best if you if you phrase it in a
way that lets them know that they can
say no I don't feel comfortable sharing
that and that's fine um and then uh
again in any area of user research the
number one Golden Rule is always always
always compensate your user research
participants they are sharing their time
with you they are sharing their
knowledge they are sharing a piece of
sort of their daily uh behaviors their
their thoughts with you this is super
valuable information uh especially in
web 3 where again there's a lot of
emphasis on the importance of data and
how we're all uh we all have the right
to our own private data right so this is
extremely valuable information they're
sharing with you um they're doing it so
that you can build a better product um
and the absolute number one thing that
you have to do is make sure that they
are compensated uh for their time
whether that's crypto whether it's you
know some um other perk that you can
offer them related to your product or
whatever make sure that it's something
that's worth it for
them I think this is your area of
expertise I mean it it can
be I think I know what I'm talking about
I don't know whether you can tell um so
let's talk about tools so all these
things you need to accomplish them
somehow um and we've got some
recommendations you don't have to follow
them exactly you can also do do your own
research see what works for you but just
so that you can have a certain idea of
what we personally prefer here here are
some suggestions so in terms of
recruiting users that is quite tricky in
crypto again because of privacy
considerations because a lot of people
are kind of anon um but there are plenty
of options so if your project has a
Discord you can use it to recruit your
user so the way I sometimes do it is
that if I'm researching a particular
feature for safe I search the feature
name in Discord and see what people have
asked questions about it and that way I
know okay this person cares enough about
this feature to ask a question about it
they are probably using it at least a
little bit they what would be a good fit
for for the test you can do a pop up in
your dab and just encourage people to
sign up to do to do research you can try
a dedicated recruitment platform there's
even a crypto specific recruitment
platform called despark you can try that
out you can also try out General user
research um platforms but they are very
unlikely to have advanced crypto users
so most most of the time you would get
people who you know have some crypto on
on coinbase have maybe a tiny bit of
experience with metamask but nothing
nothing more advanced than that um you
can recruit here like there are plenty
of people and you can get to know
someone ask them what their struggles
are ask them what they're using in terms
of wallets or exchanges or whatever the
product is that that you're working on
and um if they seem like a fit you can
ask them whether they'd be willing to to
talk to you um about your product for
getting consent uh a survey tool usually
works quite well so I can do Google
forms or type form or Survey Monkey
whatever it is that is easiest for you
to get a hold of totally fine for
conducting user interviews or tests Zoom
is is a good option because most of the
people know about it they're familiar
with it they have it set up often and
you can easily record the call
for scheduling you can use calendly
that's that's a nice option but whatever
zoom's internal schedule or Google
Calendar that also works for keeping
track of your tests and that seems like
an unusual thing to mention but trust us
you want to keep track of what you're
doing because at first you'll be able to
remember and then fast forward half a
year and you're like did we test this
feature did how many people how many
people saw this did we did we talk to
this guy he I remember his name but did
I pay him how much did I pay him I don't
remember keep track um so note who you
talk to when about what what their
contact info is whether you've paid them
and a spreadsheet tool like I don't know
Google Sheets no air table any of those
are more than suitable and then for
analyzing your notes and recordings and
kind of storing them somewhere uh is I
would say the first area where you might
want to get a specialist tool so all the
other ones I've mentioned so far you're
probably already familiar with you've
used them for other non ux
researchy purposes for storing and
analyzing notes and
recordings is it's probably worth it to
at least look into a specialist tool um
both Mindy and I use duvail and it's not
the only one at all out there but it's
quite easy to learn and it has a free
version that has plenty of features that
um should cover you if you're just
starting all right we've been talking
for a very long time and you're probably
sick of our
voices um we're going to talk some more
but now it's going to be
[Laughter]
[Applause]
different it's not over it's not
over I'm glad that someone was like oh
my God yeah they're gonna stop
talking we're gonna to be talking but
we're gonna be we're gonna be talking
directly to
you okay that was yeah oh no never mind
I I didn't like it so I so I gave up
immediately
um so no I do want to ask a question so
how many of you have a project that
you're working on right now that you
would like to do user research
with okay excellent perfect so what
we're going to do now we're going to do
a group exercise um I'm going to let you
group up with people at your table you
can group up with like the whole table
or just half the table or you know talk
to people it's a conference it has the
word confer in it you know like talk to
other people um so you know and this is
a workshop so you can Workshop things
with people Workshop things with people
are leaving they like no I can't talk to
people all the antisocial people are
leaving it's fine I would be leaving too
if I weren't up
here
ex oh okay we have we have a few
antisocial people who are staying y'all
are the real MVPs give it
up uh yeah so will ask you again to find
a few people at your table that you can
do a little bit of work with um together
with your group uh you will pick a
project it can be one of your projects
that you're working on uh one of you at
the table is working on uh or we will
give you an example project uh that you
can just sort of imagine that you work
you're working on uh and we will ask you
to create a research plan uh for that
project so that by the time you leave
here today you will have a ready too
research plan and you could even start
doing uh research on this project here
at Devcon um with uh with potentially
real users um yeah so where's the
clicker who's how it's going to work uh
first I'll ask you to again to pick a
project that you want to work on our
project that we're going to give you or
a project of someone at your table then
we'll ask you to choose uh one of the
two methods that we explained today
usability tests or user interviews which
one whichever one seems more relevant
for the project um and then we will ask
you to uh make a research plan and
Christina will be handing out um the
guidelines here for uh how to do this as
uh as they hand those out I'm going to
be uh explaining a little bit more about
what is put is is is in this research
plan and I got to do the clicker as
well so um first you'll need to think
about well first you'll need to think
about what is my project which method
are we doing then you need to think
about what are the goals of this study
what do we aim to learn from this
research is it you know people's General
thoughts about um this area maybe if
you're working on a on a Dai project and
you want to learn people's General
thoughts and like experiences with AI uh
maybe it's more specifically like I have
a project I have a prototype I have a UI
and I want to know where people are
struggling to use it um yeah so like
what are your typical goals um what are
the
criteria uh what were how will you
recruit users what how uh what will you
have users consent to how will you get
their consent how will you compensate
the participants again very important uh
and then as well to plan out the
research uh the research
interviews um and Christina and I will
be walking around as you go ahead and
get to work and answering your questions
during this process
can I just stick it like that
okay huh
um if you need to refer back to any of
the the examples that we gave any of the
information that we shared during this
talk uh you can scan this QR code and uh
find all that information uh in a medium
article as well to help you out
kind
you
okay is anyone missing these or wants an
extra
copy okay one here one here cool coming
up
ch
if anyone wants more at any point just
let me
know e
where
[Music]
s e
that's
know
[Music]
TR
e e
[Music]
body
l
w
[Music]
s
spe
t
talk
tell
e
e for
want
to
[Music]
e
e for
[Music]
to
[Music]
w
oh
[Music]
let
[Music]
the
o
[Music]
want
[Music]
so
[Music]
to
[Music]
it's
[Music]
the
hey so in about 10 minutes we'd love for
some of you to share what you have been
thinking about like things you have come
come up with things you maybe still have
questions about things you're you found
trickiest what you've discussed um so
let's take another 10 minutes and then
we'd love to hear from some of you
you
this
[Music]
you're
[Music]
don't yeah
[Music]
ask
f
[Music]
that's
[Music]
you
[Music]
w s
[Music]
no
[Music]
f
o
[Music]
do
[Music]
you
[Music]
B
[Music]
all right all right so we are really
really hello everyone Hi hi hi we're
really really curious to hear what some
of you came up with like what products
you decided to to research what
research plans you've you've come up
with um would anyone be willing to
share um what they what they've
discussed so far and and how that went
it it like this is not a quiz we're not
going to you know grade you on how well
you did but if you want some feedback
we'd be happy to to provide some um
we're just curious how this how this has
been going for you anyone willing to to
share just a little bit yay thank you so
much um I did both I did like brief in
interviews and then let the users test
out the
product I that was not the
[Laughter]
activity well I executed the
plan so it was really insightful for me
because first of all what I really liked
is that I had like two different culture
users so because that was really
interesting to me what differences they
could see on my product no and I asked
about uh first of all just so you know
my product is an nft platform that
donates funds to Environmental
Conservation projects so I got a a
really good Insight of like what are the
incentives that people have behind
purchasing nfts we also did um a lot of
technical issues that I hadn't seen
before I could see and also
like I I had a good Insight that like
foreigners are interested in Latin
American Art because it's different it's
not like something that they us usually
see on on other nfts so I thought that
was really cool and yeah thank you so
much both of you for helping me and
that's it yeah thank you I I like that
you mentioned that you uncovered some
technical issues that's kind of the
unintended bonus often of usability
testing you just you just find a lot of
bugs and often after usability test I I
have to write to some of our devs being
like oh also this happened do you know
why and they be like oh I've never seen
this interesting um so yeah devs devs
like when user research happens as well
because then you find a bunch of bugs
for them to squash yeah for a certain
definition of
like cool thank you so much for sharing
thank
[Applause]
you um does anyone else want to share
yay
awesome test test hello everyone hi my
name is tiha I'm a ux designer so uh the
project that we're going for is the beta
Basque example project so we're going
with uh user interview as the research
method and our research goal is to
understand what problem users have when
creating a wallet and for recruitment
our target audience is basically novice
users basic crypto knowledge but uh
crypto
Enthusiast and
recruitment what does it say here uh
okay how will we recruit them we have a
few options here um conference web three
event
uh Reddit forums and
Twitter and as for consent how will we
consent participants since we're using
um in-person user interviews uh we'll
ask them to consent for voice recording
note taking uh their time and also if
they're pre they prefer a one-on-one
session or um interview with a lot more
people
so can you uh tell tell me about that
with one one person or or many people
yeah oneon-one if they prefer um a
one-on-one session like a private
session or if they're open to um talking
to sharing their um details with like
other people oh oh I see so yeah for
example if we're doing it at a
conference or events there might be
people around so yeah yeah yeah that
makes sense and then for compensation uh
we're going with a few options uh either
our own token nft access to side events
yeah swags or airdrop yeah yeah all of
these are are great yeah so for um the
interview itself uh the duration will be
between 15 to 20
minutes oh and so these are the
questions that we would ask the first
one is um what are your goals getting
into crypto second which digital assets
are you interested in and then
um tell tell me about a recent weet pre
product that you use and why did you
choose to use that specific product and
yeah that that last one is is a really
really good one because you kind of
asking people to just tell you about
their recent experience recency is good
because they still have a lot of
memories about what just happened and
then it's it's specific so that they
don't have to imagine things or lie to
you just because they want to fill the
space you know so and then this is kind
of kind of like a a follow-up question
to that question briefly explain a
recent problem that you have with that
weet PR product and what do you wish
they would have done better yeah that's
all from us thank you thank
you any else
oh over
here hello I'm Hussein uh founder or
developer and developer of campus AR
which is a decentralized learning
management system s for companies and
communities uh we are the first uh LMS
on U chain uh
and uh we chose usability test and uh
there will be five scenarios and for
each scenarios we will provide them five
or 10 minutes and actually uh I will use
this
um plan for for my friends I will start
my friends in school so I I will um ask
them like uh follow the most Basics
basic flows uh for example registrating
verifying your
identity and our course modules
registrating Arc modules because it's a
bit comp maybe a bit complicated for non
crypto users because to register an arc
module you need to stake a sort of token
that's determined by the publisher of
the arc
module so
um CR criteria is uh being student
Community or company member of company
or
Community
um so yeah I I've written two uh
scenarios for now but we'll complete
them first um onboarding process explore
the onboarding process try to First
verify your identity with open campus ID
on chain verification using your wallet
or email it's important it makes so
easier uh to onboard uh for nonto users
and then uh here are the required Gus
funds let's try to to uh register for an
arc module and then third one maybe uh
go through the arc module experience and
tell me uh the difficulties or the good
parts you have
experienced uh fourth maybe
uh in each Arc modules there are some
parts that you collaborate some people
you give some feedbacks their Cotes they
give feedbacks to your Cotes and uh what
the general it's yeah it's it's a a bad
question what's the general experience
you have uh I will make them specific
it's maybe fourth one and um fifth one
maybe um in campus Arc you can register
some Talent programs called Arc Talent
program programs that's um uh organized
by web tree companies to explore the
best talents these programs uh include
some Arc modules to complete uh each Arc
modules have three or four Pro projects
to complete so uh the whole idea of the
AR T
program uh to make them make it easier
to explore
talents uh like
um there there's a proof right uh
completing uh many project that's
determined by the company itself right
so I will uh ask them to join uh AR uh
arent programs and ask some specific
experience details so uh it's all thank
you thank
you one thing I will say is that um it
can be really hard doing usability
testing with your friends because it's
even harder to pretend like you can't
help them or even harder for them to
pretend like you're not there so uh you
really have to tell them in advance like
listen during this test I'm not your
friend yeah I will definitely say that
actually uh I'm planning to um like open
the uh screen record and leave them by
theirself so uh yeah I will get some
consent about it sure okay it's an
important Point by the way thank
you I also wanted to ask that uh add
that maybe during all five scenarios is
a bit ambitious you you'll be surprised
how long it sometimes takes people to do
what you think of as basic tasks so
definitely like try going through
everything that you want your testers to
do mhm um and and time yourself M and if
it takes you someone who knows the flows
a
while budget for maybe three to five
times
longer uh for for your testers so it
might be necessary for you to prioritize
a little bit and be like okay all five
will take a while and and usability
testing is quite tiring both for you as
as the moderator and for someone who's
like oh my God oh my God what do I do I
don't understand anything um so try
to yeah I want to pick pick the ones
that are most important just focus on
those yeah I want the all of them in one
session yeah you're absolutely right
perfect thank you thank you who else
wants to sh we already have someone with
a with a mic oh excent
uh hi guys I'm PR uh I had a big team
here uh um so I have uh a few designers
here uh and deel and there was this D
guy so uh mixed bag of Team uh mix bag
um uh so um so we have an alpha product
uh we we're from 42 money and uh we're
basically uh trying to build
decentralized front ends for smart
contracts and uh um we really haven't uh
figured out um who the users are
currently like we have like we're an
impass where we can you know uh go into
different directions so we're still
trying to figure that out so we started
with usability uh testing but then we
pred to uh user interviews and uh what
we found was that uh initially um the
users were hackers def analysts right uh
they could
um you know like just prompt the llm to
be able to generate frontend for smart
contracts like directly interact with
smart contracts right so found a lot of
useful things uh from this uh from this
exercise uh for example def finalists
need a little more um interactivity for
with drag and
drop uh they need a little bit a little
more interactivity with uh drag and drop
options like components uh they need a
way to be able to
research um where the yields are coming
from you know like def Lama maybe or
something like that and uh the ux guys
uh the the they had suggestions about um
you know having the Twitter feed of um
the of the smart contract that you want
to interact with as social proof as what
is exactly going on right so and uh d
about the smart contract is also useful
uh you know like security and uh you
know risk about the interacting with the
smart contract and then uh generating
the fundr for it so yeah so basically
found out a bunch of things uh useful
and uh yeah cool thank
you anyone else wants to share yay
uh hello everyone uh we did a tiny
little bit of usability testing here so
oh yeah you also just went ahead and and
did the thing we we just did the thing
we just did the thing so um so uh for
you to understand the context um I work
at
a graphin which is what we do we do an
alternative Android if you ever used
Android there are multiple different
types of Android and we have our own
type of Android and so very recently
very recently I helped our project uh to
work on the setup wizard thing if you
ever heard about it Setup Wizard is a
very first thing a user interacts with
when they uh open up the phone so and uh
we've decided to do a tiny little bit of
user experience testing and thank you
very much for doing that uh cuz we uh my
friendly colleague uh has never ever
used an Android phone as far as I
understand right used Android okay long
time ago long time ago yeah yeah very
long time ago so
yeah no it's really good because uh it's
amazing to see uh people with different
uh perspectives on six and so uh here
are a few things that um I can say about
our setup wiard for example um when the
user first like sets up the fingerprint
uh for their phone um it's usually a
good idea to show them like a like an
animation right to show where the sensor
is but it's still not clear and the
animation might not be clear and uh or
or other stuff so like not not not
everybody knows every single phone not
everybody knows where all the sensors
are or what kind of sensors are so it's
very complicated it and so if you never
ever like so
oh I'm so sorry I'm jumping from place
to place yeah yeah but like uh it's one
of the one of the things that we'll work
on more is to show a better fingerprint
animation so that the user would know
where the uh fingerprint sensor is
located yeah this is one of the things
and I think I should probably stop now
because otherwise I will talk a lot more
and this is like the first part of like
my two pages So yeah thank you thank you
that was really fun thank
[Applause]
you anyone else
yes hey hi um so we had a lot of help
from the guys here uh actually we uh we
have a game we would like to create and
uh where they pinpointed the first uh
thing that that maybe if we think
something is an issue or a need for
users we we should we should ask them so
for example we thought that that that uh
um um ownership of assets digital assets
is a is a key question and it turned out
even in this table it's not a key
question for everyone even though they
play so uh yeah that's maybe the
colleague here phrased very correctly
that we started too technical so we
started that we must Implement an FTS we
must implement blockchain we must
Implement our own coin or whatever our
own economy and it's not sure we have to
so yeah I think I uh I learned this and
it's h back to the basics so thanks guys
thank you I just want to add it was
useful on the motivation of the game H
they they are building a web three game
so they could own the the Collectibles
so we wanted to get users who experience
spending a lot of time getting
Collectibles and H losing them because
the company goes out of
business uh I don't know if you want to
add something no dimes dim and
money
yes okay thanks that's I guess that's
all thank you
thanks um anyone else wants to share
[Music]
anything I think do we have any
questions on uh on Mir yeah so
apparently you know how at the beginning
those those of you who are here from the
very beginning uh you know how we told
you that there would be a QR code for
you to ask questions well that was a
little bit of a lie there was a QR code
and you could submit questions but
apparently they didn't go to the place
that we could actually see so if anyone
submitted questions it's uh too bad
because we didn't see them yes you have
a question yes please go for it okay I
have a question so uh you guys said that
it is important to um find the right
customer to interview with so how to
find the right Customer because uh maybe
we uh choose randomly find customer
maybe they just a small group of our
total customer so uh uh do do we have
any rule or do we have any
uh anything to Define out the right
customer
I can start so um the five person rule
is for usability testing the reason why
it typically works for usability testing
is because um usability testing is very
much based on like psychologically how a
person understands a UI how they react
to certain things um and so people who
are in Fairly similar buckets will T
typically have the same reactions and
the same understanding of those things
so if these people are already your user
users and you do a usability test with
them it's typically five people is is
okay um so that's the first thing now if
you are looking at um a larger group
like a very large user base and you want
to do like user
interviews uh or if you're just choosing
people off the street or choosing people
at Devcon you probably want to write
some criteria and that depends on really
on your business um like what what what
trying to do for your business so if
you're building a game and you're trying
to learn um you know why some people
don't play web 3 games then you want to
look for people who don't play web 3
games but who do use web 3 but do use
web 3 like why are these people doing
web 3 but they're not doing um you know
web 3 games or if you are if you have a
large you know like I I can talk about
you know my my um the the Dow that I
contribute to um we have uh we have um
an exchange platform for example and
many different people go and do swaps on
the exchange so when we're doing user
research we have to think about okay um
if it's for a specific feature then we
want to talk to maybe people who use
that feature if it's for you know if
we're interested in how to improve the
product for people who do very small
trades or for people who do very large
trades we need to look at the data and
try to find the users that do those
kinds of Trades um if we're interested
in going on a a different chain if we're
adding a new chain maybe we're going to
add Bas and we want to like understand
Bas users then we need to find people
who do like web 3 users who do swaps on
an
exchange uh and use base so like the
criteria can add you know depending on
how specific you need to be for the
particular project probably something to
add I I think you you covered most of it
the only thing I would add is that there
is essentially no way to avoid some
element of Randomness like you said if
if you just pick five of your users
isn't that kind of random and you should
have criteria you should be aware of
okay what exactly you want to test or
interview about and what kind of users
care about this the most or could care
about this the most but if you already
done that you've defined your criteria
you've searched for those particular
users there will be more than five most
that fit those and there there is an
element of Randomness and that's
okay yeah user research is not an exact
science what do you
mean I mean it's even in you know
especially when we're talking about
qualitative research when if you're
talking about quantitative research and
you're trying to get a really
statistically significant sample and
like you know look at large amounts of
data then things are different but with
qualitative research it's it's
qualitative it's subjective it's not an
exact science and some things will be
random and some things won't exactly fit
and that's just kind of part of the joy
of uh of doing this work actually you
have to be comfortable with um with
things not always fitting together
perfectly yeah does that answer your
question yeah perfect thank you anyone
else
yes hi U how to create user research on
a product which primary focus is not UI
but rather providing some
functionalities like imagine AWS without
DUI everything that it provides but
without
thei well so what can you give um I mean
when I said UI without shiny buttons
like you can still use the terminal you
can still call some apis basically you
can still use the product but there are
nothing like the login with I don't know
buttons for input Fields it's basically
pure the functionality
yeah yeah yeah yeah so so how to conduct
a research for for a toolkit for example
well again it depends on what kind of
research we're talking if it's a if it's
an interview then you don't need the UI
at all anyway um and if you want to do
some kind of usability testing you
actually don't necessarily need um you
know your typical like login page for it
to be usability testing some people do
usability testing with you know vacuum
cleaners uh we're just we happen to to
work in in an environment where
everything kind of has a certain number
of familiar elements but that's just
because uh we're all kind of in this
same context but you can do ability
research on like on on this chair if you
wanted to as long as you ask someone to
use the product in whatever way that
makes sense like using the terminal or
coding or using the developer
documentation anything that that might
be involved in that um you will still
get information on okay this is um a
struggle this is where someone gets
confused they have a certain expectation
and they think that something will
happen and it doesn't or they're like oh
I wish I could also do this so You'
still get all of that valuable
information even if there's no like is
it
realistic people yeah that's what I
wanted to ask next like is it realistic
to do it live or just tell them okay
when you are done with using it checking
the documentation and everything else
just please then provide the answer on
so so maybe you can think about what
what do you care like it's like that
they can integrate easily your product
that they think is a great product like
uh which problem do you want to solve
for them H maybe if it's a market for
that uh once you have that clear maybe
you you can design an exercise with
these people or like or like a survey
with the targetp group where you would
gather this
information does it make no I mean the
first part is perfectly clear like okay
you know what you need to do but then
when you when you create the product
like that part of testing because I mean
you mentioned videos audio that I mean
that last for an hour or two but
basically to read through the docks to
try it out it takes much more time so
I'm not sure if can like can we do it
just sit here this is the laptop please
do it I don't know 2 3 4 hours I mean do
the people test the product live for 4
hours or not I don't know or basically
tell them okay please do this whenever
it's Comfort comfortable for you and
provide us the feedback sometime in the
future so I would say that um if you're
doing feedback afterwards then an
interview is actually a good way to do
it because often you know you send out a
survey like a feedback form people don't
like forms people really really don't
like filling out forms and if they can
avoid doing that they probably will
especially if they need to like write
text oh forget it um so here you really
might want to get those people um as
close to the moment that they did use
the product as possible and then ask
them okay think
about the the moment that that you used
our our product let's say
yesterday um was there anything in
particular that you struggled with did
you notice anything missing um what was
the point where you were like oh they
have this thing isn't that nice was
there anything that surprised you uh in
a positive way so kind of just try to
get people to reflect on that experience
and and that might be a better option
because sitting somewhere with a person
for 4 hours it it can be done
but it's kind of an intense thing to do
um so maybe not necessarily the the
right way to to start you know yeah okay
thank you thank
you um and I think we are incredibly out
of time we have one minute left so
anybody has a quick question yes oh no
two people have what I mean we we'll be
still we'll be still here like outside
of of the of the classroom if you want
to talk to us afterwards but uh yeah if
if someone wants to do a super quick
question then yes yeah uh this will be
super quick yeah so basically we at an
impass where we're trying to figure out
uh with the product uh where we like we
have like sectors of users which we want
to Target and uh um if you want to
Target a specific sector then uh we need
to build the product a little
differently right and uh um we're trying
to decide which uh paths to take from uh
user experience and user design like how
do we how do we do that you're so sneaky
this is not a quick question this is not
a question we will we will answer it
outside of the classroom thank you so
much to everyone who who came and stayed