[Music]
thanks everyone um before getting into
crypto I did work in the coffee supply
chain and I've been getting a lot of
questions about where the best coffee is
in Bangkok and I'm still looking for an
answer so if anybody knows where to go
for some excellent specialty coffee
please hit me up would love to check
that out um today I'm going to be
talking about optimism retr funding uh
it's a experimental grants program being
run by the optimism Collective and as a
data Enthusiast it creates this
incredible substrate for looking at the
effectiveness of different Grant
programs um I'm the co-founder of an
organization called open source Observer
we're building a free analytics platform
and the community of data scientists who
care about measuring impact in the open
and so today we're going to be talking
about the impact that optimism is
achieving um and identifying some
opportunities to improve and double down
on where that
is has been most
effective um so I guess just a quick
check um how many of the people in this
room are familiar with retr funding
optimism have all right pretty good so I
have a few slides that will be a bit of
a cursory introduction to optimism I
think the main point that I want to get
across here is that I believe this is a
really important experiment for the
world to pay attention to and to
understand regardless of whether it
succeeds or fails in the long run
there's something special and different
that is happening here and it's worth
paying attention
to so another chart um or first chart of
the day we have a lot of charts coming
um but it has a another poll how many of
you have seen this before how many of
you are familiar with the Electric
Capital developer report have you seen
this all right not as much uh awareness
this is one of the U most looked at
reports that the crypto industry puts
out and they track a metric called
monthly active developers um they've
been putting this out for for quite some
time now it's basically indicator of how
many developers there are working across
all of crypto not just ethereum every
ecosystem is is captured here and so
what you can see from this chart is that
um things went up a lot in 2021 they
kind of plateaued in 2022 and now things
have gone down um in aggregate from a
peak of around 30,000 to about 26,000
monthly active developers by this metric
to put this in comparison I recently
heard that uh Gemini which is Google's
AI platform they have 1.5 million active
developers by their account I don't know
if they're comparable U but orders of
magnitude greater than what we have here
across the crypto
industry um here's another metric uh
this one is the amount of ventor capital
that has gone into crypto um since the
beginning basically this chart stops at
2022 but there's another number that
they care about which is the total
amount of deal flow if we add this up we
get a number of around 80 billion has
gone into the space since
2016 so I think the interesting thing is
actually when you start combining these
types of data sets uh I don't have the
ability to go in and look exactly how
they calculate these things but I can
overlay the two and when I do that I can
come up with a new metric of my own and
that metric is the dollars spent per
developer retained and what you can see
here is that for every $3.1 million that
has been invested prospectively um as
risk Capital we only have one retained
developer in the industry still here to
show for that
um so I think we can we can do better
and we hope to do better the point is
not just that um The Venture Capital can
improve um but really that perspective
funding is difficult behind every one of
these deals there's a team of
professionals that are researching these
companies and making bets and the
reality is that it's often quite
difficult to do this and there's an
acceptance that most of these bets are
not going to work out uh so perspective
funding is hard retro funding is based
on the principle that it is easier to
identify something that has worked
looking back than to predict it going
forward um vital kelp designed the
mechanism optimism has been rolling it
out and uh it's a pretty unique way of
creating markets around
impact now to be clear I don't think the
most uh die in the wool retr funding
Maxi would say that retr funding is the
only way that anything in the world
should be funded but the important Point
here is that right now we have a surplus
of or not a surplus but a large amount
of perspective funding and we need more
Retros retrospective funding in the
world to Signal what things we value and
drive more people to build in those
directions and so that's what we'll be
talking about
today uh at launch optimism allocated
850 million tokens from its treasury to
retr funding this is a big deal and they
made a promise that if you create impact
on the optimism Collective you will be
rewarded for that impact what that
impact is how it will be rewarded TBD
but that's what we're here to
discuss uh now to date optimism has
deployed around 60 million op or as
least allocated not all that has gone
out the door um which is a pretty small
share of its treasury even though this
is a large sum of tokens we're still
very much in the early Innings and each
round has been designed around a
specific set of experiments hypotheses
that they wanted to test round one I
think was completely offchain it was a
proof of concept about 50 projects no
applications the selection was done I
think by the op Labs team rounds two and
three are where this program gained a
lot of recognition they were big Mega
rounds 40 million op went out in total
more than 600 projects participated
there were 100 badge holders um also
known as Citizens uh who were voting on
projects um I was a badge holder I think
there a few in the room still but they
were given this enormous task of
reviewing all these projects and
identifying which ones they thought
should deserve a certain amount of
funding in response to the mega rounds
in in 2024 this past year um
optimism uh changed the scope a little
bit and instead of having these large
Mega rounds they move to a system where
you have smaller more tightly scoped
rounds with tighter eligibility criteria
and in those rounds they also tested
different experiments around badge
holders and guest voters and
expertise and total about 18 million has
gone out and there's more allocated um
through the end of
2024 again we're going to be looking at
the results of the program so far but
putting this in the scale of other
public goods funding experiments this is
pretty unique it's pretty large um and
my team and I at open source Observer
the fact that this was a real experiment
of significant scale was What attracted
us to want to pay attention and work at
optimism um there's a lot of economists
that like writing long papers about how
public goods funding is broken but here
was a real chance to throw magic
internet money and test whether we could
actually change the incentive landscape
um so I think people should pay
attention to this regardless of whether
they participate directly or
not but first because the only thing
that people like more than throwing
tokens around is complaining about
things on Twitter I want to address the
few myths about optimism and the program
of retr funding uh so the first myth is
that retr funding comes with conditions
the reality is the projects that receive
retr funding are getting with no strings
attached they can cash out they can
migrate their project to salana they can
delegate to governance um or they can
keep building and what we see
continuously is that most people
actually tend to keep building on
optimism uh the second myth is that retr
funding is optimism's only Grand program
this is also not true um it is also oop
sorry
it is not the only grants program it's
also not the largest grants program
optimism also runs prospective funding
uh since 2024 they've given out more
than 60 million I believe in prospective
Mission based grants and most of the
projects that receiving retr funding
have also received some form or a large
number of the products that have
received retr funding have also
participated in these prospective grants
um I think there's a uh a narrative that
optimism expects all of its teams to
work for free and pray for retr funding
is not the case there are various
mechanisms and retr funding is just one
of them and obviously the most
experimental at this
point myth number three optimism no
longer cares about public goods you've
probably heard this before if you follow
the the optimism narrative on on Twitter
um I think this narrative really took
root after optimism changed the name of
the program uh but the reality is that
there was never a formal definition of
what a public good is optimism and I
kind of agree with this definition have
always viewed it as a kind of a spectrum
and the role of retr funding is to fill
fill the Gap is identify where is the
market not effective in rewarding impact
and using tokens strategically to fill
that
Gap myth number four VCB projects have
an unfair Advantage uh we can actually
check the data on this and what we see
is that uh VC backed projects are a
relatively small number of the projects
that have received retr funding and
historically they have not performed as
well as projects that do not have um any
VC funding so at least to date VC
projects do not seem to have any unfair
Advantage even though there are some
large ones that have participated in the
rounds and myth number five is that
optimism's badge holders are super good
at deciding how much retr funding each
project deserves the reality is that we
are just humans um we have other jobs in
many cases other ecosystems that we are
that we are working on and so it is very
difficult for any one badge holder to
have a a God view of all of the projects
and what they're doing and to be able to
review each one with a level of care
that that project most likely deserves
um so I think that a lot of uh improving
retr funding is going to be about
finding the right balance of figuring
out what humans and algorithms are good
at so that we can make it more efficient
but at the same time ensure that we're
learning continuously how to improve the
allocations and make sure we get closer
to
impact so far so good now we're going to
get into the so what has this program
actually delivered on these promises is
it effective at rewarding imp impact um
or is it just a expensive well-branded
marketing
campaign uh to date optimism has
supported a lot of projects more than
700 um and there is a power law
distribution in terms of how much
funding they have received so what this
graph is showing on the xais is all of
the projects ranked by how much funding
they have received since the start of
retr funding and you can see that a
average project who's been in the
program has gotten about 40,000 op a top
one
160 and there's a few that have received
over 500,000 at the time probably worth
more than a million dollars um so this
has been the distribution so far it is a
power law it's not a kind of flat even
distribution and what you can see is
that the projects that tend to receive
more funding are projects that many of
you may be familiar with at the very top
you have protocol Guild you also have
teams that have focused on developing
specifically for optimism um like test
and prod you have core Primitives on the
super chain like um account abstraction
437 um e you also have some of the major
defi protocols like vome um and social
apps like Zora you also have developer
libraries like ethers and weim and open
Zeppelin contracts that historically
have not received any funding um or of
this scale and so this is also creating
a recurring Revenue model for those
types of projects in any case it isn't
just funding a narrow set of defi
protocols this is a pretty expansive and
diverse set of projects that have been
receiving retr
funding and anecdotally um retr funding
has had a fair amount of impact um one
project L2 be they wrote this um
beautiful letter at the end of one of
the rounds thanking retr funding in the
badge holders but also explaining how it
actually changed how they thought about
their purpose in the
ecosystem um ween ethereum news uh this
is also one that I can really relate
with um that it was kind of a
GameChanger for them receiving retr
funding it allowed them to work on this
and not pursue other things
at open source Observer um this was
definitely a GameChanger for us and gave
us the confidence to leave start a new
organization and focus on building for
optimism um so are we a few outliers or
is there some meaningful signal
here at opensource Observer um the the
project that I'm working on WE track
cohorts of other projects and
collections to see how they perform over
time and one of the things that we can
clearly see in the data if you remember
that developer Capital report I showed
at the beginning the trend line for
optimism projects especially ones that
have received retr funding is different
on the x-axis here you have time we have
a few Milestones um related to retr
funding three the biggest round and you
can see on the y axis the number of
monthly active developers that have been
participating in those in in the program
both before and then coming out the
other side and you know the line is up
and to the right but there could be a
lot of things that are explaining this
so the next thing we want to look at is
how would they have compared relative to
some kind of a counterfactual or control
group we're not in a position to
basically separate out the groups and do
a pure experimental test but what we can
do is create a synthetic control using
data about other ecosystems at open
source Observer we look at not just
optimism but a range of other crypto
ecosystems so we can create comparable
metrics for how projects in those
ecosystems have have performed in the
absence of receiving retr funding and so
through this we can see that there's
actually
a significant increase relative to
synthetic control for projects that
received retr funding from round
three excuse
me we can do this for any number of
metrics and I've only shown a few here
but this is a a comparison of onchain
activity for projects that receive
funding from retr funding 4 around that
Focus specifically on onchain Builders
and here you can see that um is a pretty
significant difference uh that has
happened both um around the time of sign
up but then after the results for retof
funding for were were announced the
challenge here is that there are other
confounding factors we had the blobs
excuse me um we had the the den couldn't
upgrade and as a result of that
transaction activity spiked across the
board so we can strip that out and in
this case um we don't see as clear a
signal as we saw previously with the
developer activity
so this is a whole line of research that
we need to do that we need to get better
at to isolate the impact that retr
funding has had on different excuse me
on different cohorts of
projects now if we step back and look at
the overall allocation that optimism has
had so far we can see that most of the
funding has actually gone towards these
two big boxes over here so that includes
a range of governance education thank
you
um education analytics projects as well
as technical contributions to the op
stack those are these two big boxes over
here and then off to the right we have
some of the categories that are probably
most critical to the success of the
super chain which have historically
received relatively less funding so
that's about 14 million for the onchain
builders and then about 4 million for
the developer libraries so I know
there's other programs that are other
rounds that are in in the works right
now to put more funding into developer
tooling but historically more of the
funding has actually gone in these other
areas and so that could partially
explain why there's been less impact on
the uh on the onchain activity than
General open source developer
activity so what's next um this is just
one of a number of areas that we are
excited to explore together with the
optimism
Community obviously there is some
quantitative and qualitative evidence
that retr funding is having an impact
but we really aren't in a position right
now to make any causal statements about
that
impact for us at open source Observer
this is a really big area of focus in
2025 we also know and you don't need to
um bring up good heart's law just yet
but we know that it's very easy to turn
metrics into targets and to game them on
the other hand we also know that to
scale this program data is going to play
a critical role and so the real
challenge here
is finding the right balance of human
human judgment and data that can improve
decision making um metrics should be
evolutionary as we learn more
information they should improve and we
should be able to back test the results
that we get from each of these rounds
against the metrics and the decisions
that we Ed to to fund those things and
ideally
incrementally improve each time around
um this is also a big area of focus for
open source observer in
2025 one of the things that we learned
from recent rounds is that technical
experts evaluate impact differently than
non-experts now this shouldn't be a
surprise to anyone um up to now the
badge holder pool that optimism has
assembled consists mostly it consists of
a mix of experts technical experts and
and
non-experts and in round five there was
an experiment that was done to actually
test the groups of experts and
non-experts prefer different projects
did they vote differently the answer
probably not surprisingly was was yes um
there are pretty big differences in how
they voted and not just on the projects
they picked but also how much funding
they thought top projects should receive
so this chart here shows some of the
most divisive projects ones that
received um a pretty big difference or
big spread between what experts thought
they deserved and what non-experts
thought they deserved um and so going
forward I think that feedback is going
to be used to improve the design of
future rounds and particularly ones that
are more technically focused ensure that
the right composition of B holders are
equipped to um you know uh vote on on
what this product should
receive um another issue that comes up
are just like the incentives this uh
this sends to different types of
Builders uh so you have some big
projects you have some small projects
and when you have an allocation which is
not um like a very steep power law
distribution what you can what you can
see is that smaller projects there might
be an incentive to create smaller
projects or to atomize your work into
kind of a a set of very small individual
contributions um as opposed to capture
the collective impact of of a larger
team we can see this at least in in
round five where some of the projects
that did the best had the smallest team
um per on a per contributor basis and so
this could create some kind of perverse
incentive for teams to fractionalize
their work as opposed to representing it
as one cohesive um one cohesive team uh
trend from protocol Guild a very nice
essay pointing out this fact and doing
some
comparisons um and then the fifth one is
that there is a trade-off between
mathematical accuracy and just The Vibes
and the spirit that comes with each of
these rounds um we quants we tend to
suck the life out of things and there's
something very special about the spirit
and The Vibes that are created around
each of these retr funding rounds and so
as the program becomes more accurate and
predictive um there could be a trade-off
in terms of the uh the energy and the
attention that this gains um because
there's less speculation and less hope
for some for some Builders we definitely
saw that round three kind of marked a
high water point in terms of the social
media attention at least that retr
funding
received um we believe that the upside
to figuring out these things and working
on these challenges has a potential to
be huge if you just look at the size of
the number of developers that are
contributing to open source is a global
movement and If This Were a nation it
would be as large as the population of
Thailand or Mexico or Germany so there
is a large Global borderless movement
around open source and historically we
have not been very good at funding it If
This Were a nation the amount of funding
that it is allocating towards its public
infrastructure would be on par with Fiji
and swasiland pretty pretty low down the
ranking so we need to do
better
um we at open source Observer um are
very excited to work on this problem
we're hoping to attract more data
scientists to care about the
intersection of public goods and impact
and data science and if that speaks to
you we'd love to hear from you you can
scan the QR code to get in touch um join
our Discord and for any builders that in
the room optimism is having a seventh
round of retr funding it'll be starting
hopefully um before the end of the year
you can go to retrun optimism. get all
the details there and sign up thank you
very much
awesome Carl so you know next retr
funding it's coming close so take the
opportunity questions that you have for
Carl let's start that way meanwhile
please use the QR code to place your
questions uh to e so go ahead hi Carl
Thomas here hey uh thanks for the
presentation I'm uh just want to say
first of all I'm a huge advocate for
open s so far I use it all the time my
day-to-day activities always always
using libraries Frameworks which are you
don't even know how open source but are
right um so I wanted to go back on two
slides that you had uh previously me
mentioned um one of them was around the
development onchain activity you don't
have to go back but it's fine um I was
wondering would you I I know you said
there were a handful of other metrics
while you were speaking but you um
outline these two wondering would you
say these are the most important metrics
for measuring an inval evaluating the
impact of funded
projects um no I would say that they are
good trailing metrics so ideally if you
have if you're doing other things well
then you'll get more developers you will
get more transactions you'll get more
users but uh you probably don't want to
fund these things directly if you were
to just fund People based on the size of
their team then you get very big bloated
teams you wouldn't actually get the
impact that you care about um on the
other hand they do probably correlate in
many cases with like at a high level the
growth of an ecosystem um so we don't
really I I'm not in a position to say
there's there's one metric where you
should just find this thing and I think
that's that's the whole point is that we
need to test these things out in the
wild we need to combine them in
different ways and then I think over
time we'll see some kind of composite
set of metrics some which are really
good as these trailing indicators and
some which could actually be more
predictive of a future success yeah big
big search space and just a followup
question how much how much time would
you reckon that would you would need for
that for for which part for like to eval
to you you mentioned you need like your
Gathering data and your your to make an
inform well-informed decision how how
much time do you need to make that
decision um we see this much more as
like a community uh a challenge for the
community so we can definitely come up
with metrics ourself but increasingly we
want to partner with with data teams um
you know projects themselves people that
have an intuition about what metrics
they think are most useful and test
those out experiment with them and you
know everything all the metrics that
we're building are are fully open so
people can come and and test them out
directly directly thank you
yeah now we have some questions here one
is what if what are the opportunities
for data scientists to get
involved yeah so um scan the QR code we
have a number of contribution
opportunities some are pretty basic just
like helping add update um process data
what we're really looking for help is
people that have the ability to do kind
of modeling um so you know strong python
skills they're able to do things like
the the causal analysis that we showed
um they're able to process uh you know
help help us connect new data sets and
process larger and larger amounts of
data um anyone who has the ability to
kind of formulate research questions and
then design a study around that would be
super welcome um but also people that
just want to uh you know create front
ends and dashboards on top of it there's
a lot of of cool things you can
do great awesome and for for how long is
optimism going to be running retro PGA
fronts um I think the answer is probably
forever that would be the that would be
the goal um certainly the the plan is
that once the the treasury the initial
treasury runs out that there's a
continuous source of funding that will
come in from sequencer fees or from
other sources of of onchain Revenue uh I
think the goal eventually is that
everything that is happening in optimism
gets funded by retr funding whether that
takes five years or 10 years remains to
be seen um but the goal is that this is
a system that that runs
forever awesome Carl appreciate it and
appre appreciate all the questions you
have we just run out of time so please
give a great Applause to Carl thank you
everyone and
optimism thank you so much
[Music]