cool so how's it going my name is Kevin
today I'm gonna talk about guerilla user
research tactics to get mainstream
adoption for your depth so the original
version of this presentation actually
focused a bit more in the beginning on
some of the user research I've worked on
globally at Facebook and courts and some
lessons to be derived from that for
blockchain and payments but after doing
some user research of my own a lot of
people said that the second half of the
presentation just the actual tips and
tactics were the most useful part so
this presentation just gonna focus
entirely on a bunch of things that you
could do for your DAP starting tomorrow
to help validate the idea for your DAP
and the concept for your DAP and all the
way down to launch so a little bit about
me first I've touched user research and
data from a few different perspectives I
cut my teeth spending summers at Google
as a product data analysts running a lot
of Python data jobs and analyzing user
data for the Geo commerce team I spent a
summer at Microsoft managing user
research for Cortana then spent a few
years at Facebook working on growth
products product marketing and helping
drive user research initiatives both on
the product and perception and sentiment
side but also across a number of
different continents and currently I'm a
product manager quartz work focusing on
memberships and subscriptions and doing
a lot of scrappy scrappy startup
research so I guess you know what is
user research instead of a dictionary
definition I think I'll just give you my
own colloquial definition which is a
bunch of ways that you could interact
with users potential or current users
that help you understand how they do
things and so these ways or research
methods you could kind of cut in a
variety of different ways you could
think of the methods in terms of what do
they track how do people think or how do
people actually behave you could do it
by the phase of the development cycle
that you're in and which tactics map
best to that you could also do it by the
context in which the study is actually
taking place don't worry you don't have
to memorize
another main split is qualitative versus
quantitative and for those you didn't
know a lot of user research is
considered quantitative for example for
surveys especially when you do them at a
really large scale you need to you you
need to calculate the margin of error
and make sure you have statistical
significance and your findings and
results so what is guerilla user
research that it's basically just a term
I made up and what it is is it's being a
it's trying to do user research for you
tap in 2018 it's scrapping and clawing
to get any sort of validation for your
DAP that probably if it has launched an
on main yet main that probably doesn't
have a ton of users while dealing with
few resources and the dozens of other
barriers that you face as a blockchain
founder in 2018
so this is just a quick thing I drew
where this is the preferences of a real
kind of person with an actual problem to
the right is the solution that they're
using now they like it slightly to the
left is your blockchain based solution
farther to the right it is the more they
want the solution so you've heard the
saying that you need to be 10x better
than your competitor in order to get
over switching costs in order for the
person to actually switch over I've
always thought 10x was kind of a lot but
let's charitably say 2x so you're
already starting off at a huge
disadvantage you're already twice as far
away as you were before just by being a
competitor then you introduce the other
things that are happening you know
people are skeptical of crypto and
blockchain because the market crash
people may not even know that you could
build apps on block chains all the
standard exchange and meta mask barriers
all the UX and copy confusion that has
been very well documented and talked
about I'm sure covered very well by a
lot of presentations here and also just
you know if you're a platform that
requires regular other people like
businesses artists musicians you're
gonna have to convince all of them to
get on and that's going to be really
really hard
and if you're an enterprise product you
have to convince at least ten different
stakeholders within that company that
this is real and valuable and secure
which is gonna be very very hard and so
you're basically off the map now that's
kind of your starting point as someone
who's building gaps today that are
competing with actual centralized
solutions and even developers and those
who invest in them don't use DAP so this
is from a great medium article I read I
thought it was pretty funny because it
seemed pretty accurate that develop
developers who built them and the
investors who invest in them they just
don't even want to use them because it's
so annoying and hard to do and they have
lots of other busy things to do in their
life that's what we're dealing with yeah
so great so some initial questions you
may have thus far so isn't user research
only a thing that big companies and
fancy design consulting companies can
afford no I mean obviously it helps to
have money as you could see I've been on
both sides of it so I've seen firsthand
but the problems are kind of different a
small company you're just trying to
validate your idea at the beginning
you're just trying to find any time to
do user research whereas a big companies
you're more kind of launching new
product lines kind of protecting
existing products to its competitors etc
how many of you guys work for a company
that's above that maybe has more than
one full-time user researcher by chance
well I'm guessing most of you people are
consensus or perhaps coinbase or status
which is you know which is good it's
like I think we should all aspire to do
that to be honest but I also know that a
lot of you guys are really small and
just trying to hack and get something
out the door so these tactics hopefully
should be pretty helpful all right well
we're too small we're short on funds we
can't hire a researcher well so you
don't you may not actually need to hire
a full-time researcher so in terms of
finding your researchers so one
aetherium token means you don't have as
many resources for means you have more
resources so when you're pretty small
five people or less it's gonna have to
be you and that's okay because a lot of
the tactics that you'll actually be
doing like interviews surveys
usability tests which are just you know
kind of walking people through your dap
and seeing how they respond and giving
them direction those are fairly
accessible to pick up with some study of
the methodology also allow some of the
best people in the world at user
research actually started from a founder
point of view so you're set up pretty
well when you get to maybe like ten
fifteen twenty twenty-five people
probably want your designer or PM you
hopefully have a designer to spend part
of their time doing that it maps pretty
well to their skill set and if you're if
you're really in a crunch i think two
other skill sets that could map well to
user research are one maybe like a
marketer that's really interested in
marketing research and really cares
about the user or perhaps someone on the
business side that used to be a
consultant that cares a lot usually the
kind of research a lot and deliver
actionable insights within a time
constraint kind of method helps
consultants transition over and if you
have a few more people you might be
ready to make your first full time hire
so obviously it would be nice to get
someone with both the academic
background usually some grad degree and
one of the experimental social sciences
and also the practical background having
some experience at a tech company but
that's pretty hard to find my opinion on
two kind of requirements for a full-time
research hire that you're making if
you're on a bit of a budget are one make
sure they have some sort of UX portfolio
you know similar to designers a lot of
aspiring UX researchers often put
together a collection of articles or
they walk through the ideation process
they walk through user interviews they
did they document you know their thought
process and that's helpful just because
you could actually evaluate their
aptitude and also because it's just
proof of commitment as opposed to just
some theoretical thing they're like hey
i'm like a product manager and i might
want to do research you know convince
you in the interview and then also some
professional work experience i think
would help just because you need this
person to execute for you now or your
dap might be done
and often people who have been in a
professional work setting are just more
familiar with the types of constraints
that they have to work under it's not a
hard rule but those are my personal
opinions and then finally if you have
the money for example if you have ico
funds and hopefully they haven't been
siphoned off through a smart contract by
anonymous CEO you I think it'd be good
to poach an experienced user researcher
from a tech company or design
consultancy especially if you're doing a
lot of upfront idea validation and
generative research because in terms of
evaluating concepts from scratch it's
really hard to beat a lot of actual
practical experience all right so okay
we could find someone but it's expensive
still I don't have money for all these
things that you need for research well
you really don't need much so this is
just a quick example of some of the
tools and pretty much all of these are
free a lot of these are even just Google
sweet products that you could use for
free we at courts use a good chunk of
these and don't you don't have to like
memorize these I'll find some way to
chair slides after but I think the idea
is that there's really nothing stopping
you from right after this talk setting
up your own user research stack it's
just screen sharing video conferencing
survey tools actually scheduling is the
most is one of the most important ones
here if you share your Google Calendar
link the other person can kind of ask
ask you for a meeting or propose a time
and scheduling actually is a big barrier
but there there should really be nothing
stopping you in terms of actual tools
from doing an a user research and this
is just an example like when I sometimes
when I do mobile remote usability tests
I just plug in my I ask them to plug
their phone into their Mac if they have
a Mac to clicks and they're mirroring
their phone and sent aeneas ly using
QuickTime on their laptop it's in their
screen sharing alright great so oh
that's great but I'm also just I'm just
a developer I'm just a founder I don't
really know much about what to actually
do do I just talk to people well that's
what we have the rest of this
presentation for a bunch of things that
you
could do at each stage of the journey
and I might optimize a bit more just I
might go a little fast just to make sure
I cover everything but at the end I'll
make sure to provide my contact
information and we could kind of talk
through anything I'm happy to answer any
questions so first you just want to
start small not big I want to make the
distinction between protocols which I
think is good to thank super super into
the future
top-down house it's gonna survive in 50
years governance that's great but if
you're a DAP which is just an app with
some smart contract component
you should smart small just like regular
tech companies pretty much every giant
tech company started with a super small
slice of a pie Airbnb literally was just
air mattresses at just conferences then
went to just air mattresses then went to
you know renting people's apartments now
they're doing luxury travel and
experiences and concerts same with uber
started with just luxury limo cars then
all types of cars then autonomous
vehicles and the reason why that is is
because when you win a small slice you
get a unique vantage point into the
entire kind of pie of that industry as
one of the winners to see the other
opportunities in that pie that no one
else can you've learned the best
practices you've learned the industry
better than everyone else you have a
well-oiled team with great operations
every one of your people in their field
is an expert in their particular duty
so you're just set up really well to
dominate more and more of that pie and
that's kind of the mentality that I
think would be beneficial and just a
quick example so this auction / - I know
I don't know him very well honestly he
actually was never really a crypto guy I
think he he's in his mid-20s and it's
old e-commerce startup - Ebates for a
modest sum beforehand but it's a super
simple experience where you just
download a Chrome extension and if I
want to buy something at one of these
retailers and there's a lot of retailers
covered a little pop-up comes up I click
it great it's activated I go through my
normal checkout
floo and then two days later I just had
Bitcoin in my wallet and there they have
custodianship of the wallet and I
believe you could withdraw after a
certain amount to either a bank account
or your like a like a Bitcoin wallet but
I don't know I mean it's this is I think
a good example of starting small and for
what it's worth even though it doesn't
matter too much they have a lot of
strong investors on their side a good
amount of funding and they seem to have
good adoption so I feel like you know
there's really nothing separating any of
you from being able to just really
tackle very real problems yeah you want
to do lots and lots of interviews up
front depth is more important than
breath I think and you don't have to
quite go so far but I did want to
highlight a cool example that I saw of a
stable coin company sello who went all
the way to Kenya to do user research on
the ground to understand how Kenyans use
paisa which is mobile money that they
could send really easily over your phone
to other phones it's extremely extremely
widespread and popular in Kenya and for
example I thought an interesting insight
was empezó kind of spread adoption wise
not by people looking it up and you know
kind of trying to get it or download it
or whatnot but through other people in
the village that people knew teaching
them physically and showing them how to
use in Pesa it was a very like local
growth that way so yeah just sometimes
you really need to just talk to people
to understand the nuances that really
define the ways that people think and
act about certain products and the tough
part of this stage especially for adapts
is that at this point you probably have
the least resources like even if you're
funded this is probably a pre funding
point but research is the most important
stage at this point just because of the
large amount of time it takes to create
a set of secure and well tested smart
contracts so it's kind of a lot of
pressure
but at the same time a lot of the tools
that you're gonna use are relatively
straightforward for you to pick up and
as long as you focus and try I think
that will be a big improvement in
self so when we talk about design and
prototyping I actually went to a pretty
great workshop yesterday by these
designers named Samara and Hanny who
walked through a few great prototyping
strategies but you basically want to use
interactive prototypes or kind of a web
mobile MVP no blockchain at all just use
a centralized server with some dummy
data just to get something in front of
people that resembles what they would
actually be using so if people didn't
like your idea in the interviews you
could show this to them and this should
be a fairly representative sense if this
is a real problem that they're having or
not and you want to get as many users as
possible on it
so briefly walking through an example of
what exactly usability test is for those
of you who don't know
so at quartz we launched this product
which was a story series with 180 degree
video called machines with brains and we
decided to be pretty experimental in
terms of launching it you know you kind
of scrolled down in your article amongst
different cards to read your article and
swiped left and right to get to other
stories we thought it was intuitive but
we also had a big time crunch and we
heard some bad anecdotes upon launch and
so we conducted some quick usability
tests we found that the large blank
spaces at the end of cards made people
think that the story had ended and so
they just left we also found that
scrolling card to card because they were
different heights was not very intuitive
and people felt like they were there
scroll was getting hijacked and we also
found that people were accidentally
swiping horizontally when they might
meant to be swiping vertically probably
because they were swiping at a bit of a
diagonal angle and so we quickly kind of
enacted fixes to these solutions so
usability fixes these types of things
can really help you in terms of your
product you get a lot of great
information up front you want a design
for the least blockchain savvy and
technical version of your user possible
and so I actually wrote an article about
this a while before the use case we did
a hackathon the problem was build a
farm-to-table
local supply chain DAP built it on hyper
ledger and they mentioned that one of
the users was this old 70 year old
farmer that doesn't know how to use
email so we kind of designed with that
in mind made it super simple this
applies to really everyone like if
you're building prediction markets you
probably want non blockchain experts to
use it at some point so you should
probably try to optimize and design a
bit more for like a regular kind of
gambler even if you're building
developer products
you probably want developers who don't
know that much about blockchain now to
kind of on board onto blockchain and use
your products so you want to make it as
block blockchain jargon free as possible
at least in some of the main areas just
so it's really easy for them to on board
since actually a lot of develop there's
a lot of really skeptical developers for
blockchain and crypto out there and even
crypto products honestly there's a lot
of people who trade crypto and on crypto
don't really know how it works so you
just always want to optimize for a
little less technical and a little less
blockchain savvy than what you might
expect and yeah so you you got a you got
a test your copy both product copy and
marketing copy for product copy Prada
copy is always not as clear as we think
and don't use lorem ipsum use real copy
in the context of the product from
marketing copy even for like developer
products so you know these are some
really complex developer products with
some really really simple marketing
messaging stripe watson Twilio I know
we're all proud of the technology we
built and want to show it and you also
want to impress other engineers and
investors but especially on your main
homepage
you know permissionless decentralized
credit market algorithmic efficient
money markets it's just kind of not that
necessary
you're just not optimizing for as many
people as you could be even financial
products you know fidelity it's not like
they're fast-moving startup credit karma
wealthfront you notice that they're
focusing on value propositions like what
the end
results of what this product will do for
you how you will benefit as opposed to
here all the small tactical things that
we have and you know for marketing to
global audiences I'll just kind of let
this slide speak for itself but please
just you know even just through some
through the crypto Network or just talk
to a few people translations can go
really wrong so yeah just to talk to
people and I would highly recommend that
you don't move past this phase unless
you have users in the bank and what I
mean by that is having a bunch of users
that are kind of you you interview them
you're do testing with them they're so
into the idea they immediately see the
value this would bring to them they're
bugging you every week asking you when
your products gonna launch they're
talking about other people kind of in
their field that meet their profile they
really really want it and you kind of
have to be honest with yourself here but
once you get into the smart contract
development phase you start to get to
some you know really heavy upfront costs
so you really want all of the work or
most of the validation work to be in
these first two steps and then when you
get to the test net phase you know test
that's mainly just to test out your
smart contract functionality by this
point your idea and your actual product
experience should have been vetted in
the previous steps you will want to make
sure you just explain in a simple way
what a test net is to all users even
though it has a word test in it it's not
that obvious exactly how the components
work for example on meta mask on your
test set it still shows a dollar value
of the ether that you have in your test
wallet which is kind of confusing
because when you're looking at a wallet
you usually and you see money here's
like oh is that like
money that I have just make sure that
you explain what it is I would recommend
that you have basic event tracking like
the funnel major actions user sessions
Google Analytics is free you could just
use that there's a ton of documentation
and if you really care about user
privacy just don't associate sessions
with any particular user identifier I
just I'm not sure how you could realize
if you've hit success or realize how
close you are to success without
tracking basic metrics and when
I have is too and that we've you know
I've seen a lot is to use support
messaging products on your DAP we're
actually in the middle of installing one
at quartz right now you know I'm sure
most of you have seen this this is
intercom you see a little chat window
you click it it pops up you could
immediately start chatting with customer
service or you could read a few
different resources I think this is a
great way of just setting kind of a set
it and forget it automated way to catch
people and who go to your DAP and get
some feedback from them because you
really remember this is guerilla user
research you need to use every tactic
imaginable at your fingertips to try to
get any any feedback at all when you get
to the main net launch another example
of an automated feedback tool that you
could use our in-app surveys so in-app
surveys actually in our comm has a
plug-in for in-app surveys there's some
other vendors like Survey Monkey and
CIPA et cetera just a quick sample that
you could do you know hey this is really
important
how'd you find out about this step why
are you interested in trying it provide
some free forum feedback options and
then collect their contact information
for follow-ups just tell them that we
won't associate it with your wallet your
you'd like your product data but you
need the ability to follow up with these
people
to really understand some of the use
cases or pain points or problems this is
probably like the most relevant set of
users you could get information on so
you really need their contact
information obviously you could go
really extreme so it's well-documented
that Facebook would send in apps would
show in-app surveys or web surveys
asking people if they cared if they
thought the Facebook cared about its
users it is a bit of a blunt-force
technique but I have to say I you know
while is there and looked at a lot of
that data and it mapped pretty well to
how people felt about Facebook so it was
valuable be desperate for feedback
aggressively scour the web social for
anyone who has used your products
similar themes don't be too proud to
pivot at this point if you realize that
your idea isn't something that people
want you've already learned a lot about
how'd apps work how blockchain works be
honest with yourself don't do that you
know everything else is great but the
throughput so I'll just wait two years
for sharding or it's just meta mass
that's a problem I'll just wait a few
months for delegated transactions for
more people to download wallet connect
you should probably be solving a problem
that in spite of everything that's
happening right now keep some people at
least won't really really want to use it
so if it doesn't meet that bar I would
just really really stop so that's that
here's some of my contact information
any questions I'm seriously happy to
answer them there's also just a lot of
talented people at this conference that
could answer some of these questions
better than me
I mean yeah I could sense lights anyone
or I could find a way to post them okay
thanks everyone for Kevin
[Applause]
you