foreign
[Music]
we have two general methods for
conducting user research which is
gathering feedback on a digital product
with the intention to improve it so we
have qualitative and quantitative
qualitative research involves conversing
questioning and observing users as they
interact with your digital product
this usually comes in the form of
interviews or live tests
it's also called participatory design or
having your users participate in your
user's development or at the products
development sorry
and on the other hand we have
quantitative research which is capturing
user activity patterns and behaviors
with a product using data or Analytics
also I'm going to stop using the term
users for the rest of this talk because
I think it sounds pretty detached and my
hope for web 3 is that we can stop
defining our community members by the
loan fact that they use the product
and start to see them in the broader
context of their complex lives and their
complex needs
so we're learning about people to better
serve them with our digital products
right
let's unpack those two methodologies
starting with quantitative
some benefits to quantitative research
is that we can determine patterns to
investigate further
so data can reveal what's going right or
wrong based on our products goals it can
also reveal insights that leave us
curious
some drawbacks to quantitative research
is that the data doesn't tell you why
something is happening and to guess why
is fine hypotheses are really important
but it's important to treat them as
untested subjective results to look into
further
another drawback is that people are
conscious of their privacy so a lot of
people in the web 3 Community use Brave
or vpns that make their actions off
chain untraceable
so this means that the data that we pull
off chain may never be the full picture
of community usage
data collection and Analysis is
expensive and time consuming
and it can also be unethical
so a person's data can be used to
improve a product and increase a
company's profits if that data has value
and the person who is creating that data
is not collecting any of that value is
that ethical
maybe if they sold it to you or if they
agreed to give it to you without
compensation but that's the argument
here that to collect somebody's valuable
data you should either compensate them
or gain their consent
data can also be biased and there's a
lot of types of bias that can occur here
but the most common is called
confirmation bias which is when somebody
is interpreting the data set and then
comes to conclusions based on their own
ideas or interests
what can also happen here is the
manipulation of data visualizations to
further justify that bias and present it
as truth
moving over to qualitative research
so some benefits to this is that it can
validate those hypotheses you made about
your data
so you can get to the bottom of Why by
asking people by watching them interact
with your product and asking them to
think out loud it is super effective
you're also compensating people for
their time and gaining their consent to
participate in your study so with those
two criteria you're becoming your
research is becoming more ethical and
legitimate
it's also encouraging Community
Development and responding to that
eagerness to contribute in web3 so
people want to be involved so let them
reward them encourage them this builds
loyalty and buy-in
and you're building empathy with your
community by talking to them by seeing
their faces it's really important to see
your community as humans and not data
points and it's really easy to detach
yourself from this valuable concept if
your data is the only source to
understand the people that you work for
so it does have some drawbacks
participatory design can be really
challenging in an online International
Community
many people don't feel comfortable
sharing their identities they don't want
to show their faces on camera they may
not speak your primary language these
can all be really intimidating barriers
and there can also be confirmation bias
here so it's important to keep in mind
that this line of research is informal
we're not using rigorous controls under
the scientific methods This research is
always somewhat subjective and open to
biases
so my Approach for participatory design
in web 3 is always changing but here's
what's been working so far
so I found that by offering tiered
approach for users to participate it
lets you meet people where they're at
so you can have these bottom tier
rewards for submitting to long-form Tech
surveys
this lets people translate them and they
can also remain anonymous
you can offer mid-tier rewards for
having people record looms of themselves
screen sharing and performing certain
tasks
Um this can also be translated this can
also be anonymous and they can speak at
their own pace if they choose to
and then you have a top rewards tier
where you have a video call and they
perform certain tasks answer questions
and this works best when you can both
communicate freely with each other
systems like this are best managed in
Discord or other community tools like
crew 3.
ultimately I advocate for a
participatory design focused approach
with data to guide you towards those
areas of improvement
I think there's a really beautiful
opportunity in web3 to practice genuine
empathy for our community by working
alongside them
thank you
thank you what kind of tools do you use
on the quantitative side for user
research
on the quantitative side at the moment
we typically hire somebody to do that
for us so we've been working with arcx
who've been really great at accumulating
data for our products
on the smaller scale side we use Google
analytics that's easy for me to
interpret but yeah for the bigger
questions we Outsource