foreign
[Music]
I work at optimism I focus on security
in our protocol I previously was an
auditor and co-founded diligence with
Gonzalo and have had the Good Fortune of
working as a client with these three
gentlemen and not yet sir Torah but have
had an opportunity to get to know them
really well so I really feel in my
element and I'm excited to have this
conversation so I'll let them introduce
themselves quickly if you could you know
give us your names and tell us about
your firms and what makes you special I
can go first hey guys
um and thank you Richie yeah if you're
watching
um yeah we co-founded consensus
diligence almost six years ago now or
going to that
um me and John who's now at optimism
unfortunately and the consensus
diligence is a crypto native security
firm that has for the entirety of its
life done Security Services
and has also a product side which you
might have seen so mythex and more
recently diligence fuzzing and a bunch
of other smaller tools that are all open
source that you can check on our website
hi everyone my name is Chandra and I am
from sartora sartora is a automated
auditing company you can think of it as
that way what we do is we use formal
verification techniques to prove certain
properties about your smart contracts
and I'm personally more on the r d side
and one of the things that we are
focusing on these days is how do you
have more Trust on the specifications
that you write because at the end of the
day that's really what we're using as
the ground Truth for verifying your
smart contracts so if you have any
questions about formal verification I
would love to hear about those everyone
my name is Maddie co-founder and
director of Sigma Prime we're in
information security consultancy focused
on blockchain technology doing a lot of
smart contract auditing as you can
imagine but very comfortable diving
deeper into other layers of the stack as
John mentioned layer 2 systems new l1s
and whatnot we're also the founders and
maintain owners of Lighthouse the rust
implementation of the ethereum consensus
protocol some of you here I'm sure are
running validators some of you may be
running Lighthouse as well we had the
merge a few weeks ago my six milestone
for everyone massive milestone for Sigma
Prime and yeah hi everybody I'm Jonathan
Alexander I am CTO at open Zeppelin open
Zeppelin we are a security advisor
security auditor we also work on open
Zeppelin contracts
open Zeppelin Defender and we are core
development contributors to Florida and
hi everybody I'm Nick Selby I'm with
trail of bits we are an audit and
research company based in New York but
people are all around the world uh in
addition to audits we have a research
group that that mainly works on just
sort of pushing the edges of of a bunch
of different Technologies including
blockchain uh and where are things going
next and we make some pretty popular
open source tools like Slither and
Echidna which probably several of your
you know familiar with thanks I'm glad
to be here
awesome thanks everyone uh so first
question I think is like maybe a bit of
a softball just to get us going
uh and I'll I'll pose this to Medi first
and then just let the conversation get
going
so it's been three years since the last
Devcon what do you see as the biggest
changes in that time maybe it's
something you're excited about something
you're worried about we'd love to hear
it yeah interesting
um let's see I think what I've been
noticing is the quality of the projects
that we've been reviewing has somewhat
improved over the past three years uh
this caveats obviously we still get the
occasional you know D5 projects with you
know lack of access control and the
typical unsafe and safe external calls
and re-entrances but I'd like to think
that thanks to some of the tooling
that's been available it's been doing
great work on that front developers have
a lot of abilities these days to pick up
the low hanging fruits contracts are
getting more and more complex in the D5
space particularly some of the lending
protocols that we've been reviewing are
pretty hectic but yeah yeah I think you
know the amount of money getting stolen
on a weekly basis these days is probably
greater than what it was three years ago
I'd say but it's probably due to the
fact that there's just so much more so
many more people getting into the field
a lot of new projects being deployed and
obviously you know our time is you know
constrained we can't unfortunately
service everyone all the time
um but yeah so goods and bads I guess
we've seen a lot of projects come and go
too and a lot of money come and go so
yeah I think that like even I fully
agree with Maddie the quality has
definitely improved in I think the bear
Market kind of helped at that okay
anybody have uh something different to
say than better quality I have a mixed
bag all right so like
three years ago it was it was a simpler
time uh things were a lot easier to look
at things were a lot more self-contained
uh and as you just said right the the
complexity the the Maze of dependencies
the uh the the complexity has kind of
exploded at the same time that there's
more and more value so the stakes are
much much higher I think on the other
hand you're right the tooling has gotten
a lot better but it's also I think all
of us here working with with firms that
are we're getting used to the we're used
to the risks we're used to the threats
we're we're inured to the patterns we we
can see them a lot more clearly
everybody here has a lot more experience
in in finding things uh and on the other
other hand
um three years ago I remember being
describing to board members of a of a
pretty large crypto company uh about
nation-state threats specifically
talking about North Korea and they kind
of looked at me with blank stairs and
they were like well you know so what and
I think that over the past three years
that that nation-state involvement has
gone from a kind of a risk to a definite
and definable threat
yeah
uh Chandra do you
so well this is my first Devcon so I
wasn't there in the the first the one
three years ago uh but what I've noticed
is that
um there seems to this is a very fast
moving field and there's like a new
terms new buzzwords new Concepts that
keep getting thrown around a lot uh but
I think you know the main thing that I
would uh you know say here is I think
it's important to remember that at the
end of the day uh like companies like
surtora or like you know formal
verification companies or like any
company that is building tools for uh
verifying smart contracts they actually
rely on a very fundamental uh concept
and I think that even though this field
is very fast moving uh the fundamentals
are still the same and you know those
ideas uh go really far and I'm actually
excited I think that we should be able
to scale some of these techniques uh
pretty well to some of the more recent
uh
Concepts that are coming around in this
field nice okay I'm gonna I'll give you
the next question but let's we'll keep
it moving uh so
yeah so
we're done with softballs
um Jonathan what responsibility do
Auditors bear when code they've reviewed
has been exploited
thanks for Daisy hey
nope that's targeted
um you take a sip of water for yeah
well I I think
um
first of all I think
all of us because we uh I think
everybody up here we take our work so
seriously that the first order of
responsibility of course is dealing with
the issue that's found and I think that
I know we at open Zeppelin make every
effort if
we've had situations I'm not going to
say we've had situations where things
we've missed have been found but
customers we work with maybe have a
situation arise and you are a trusted
security advisor to them you're going to
Rally to help them in the situation and
that may be in discussing what are the
potential fixes in reviewing fixes and
even talking about public Communications
and that is a uh I think if you are with
a trusted you know we view these
relationships as Partnerships then
that's what what we're going to try to
do
um I think that in the case in a
specific case where something is missed
in an audit which sometimes
um like we've had some situations where
uh we'll we'll do an audit and then
before thankfully pre-release an issue
is discovered that was missed in the
audit maybe because there was uh
another you know set of eyes on it or
maybe there was internal testing and in
that case uh we always view our
responsibility is to do a very thorough
postmortem
on what happened
exactly what was found have our team
analyze itself as well as get other
people in our team and other get the
customer involved so that we can all
learn from that and figure out and
explain to ourselves and improve our
processes going forward so I think
that's the very common uh approach so so
we've heard from Jonathan that it's I I
think this is just like late to the
preface on the same topic but Jonathan
has been open that a top-tier auditor
can miss something
and so you're lucky right like it got
caught before it hit production that can
happen to anyone so let's like put that
out there is anybody on the stage like
actually had something they've added be
exploited in a significant way I'm
curious
not really but we've had stuff being
disclosed after it's in production yeah
like I I think that like we
establishing and we've said this many
many times we're not insurance companies
right so that's the Baseline but there's
an ethical and moral code that like
that we uphold to and so if something
happens obviously this is also Case by
case on a case-by-case basis it should
be analyzed differently because if it's
if it's our
the if the problem was created by us
specifically then we might react one way
whether if if the if there it was just
like a Miss we might react some other
way but at the end of the day
we'll try our best but we're not
obligated to anything right nor can we
be because that actually destroys the
entire relationship
well I'll go because I'm just gonna just
jump onto it uh we we actually don't
I don't think that we have a different
way I liked what you said about the
moral responsibility I think it's really
it's really important um I I don't think
that we have a different
um process we have we have a pretty
stringent process to to determine
um
whether whether something was missed but
but much more important is what happened
and how can we help and how can we reach
out proactively they're going to be an
incident response mode they're going to
be their hair's on fire there's going to
be media involvement they're they're in
a world of hurt a lot of us at trail of
bits have worked in the incident
response field so we're we're very
accustomed to understanding what they
need to do right now uh to stop the
burning to stop the leaking and and get
things done so our first reaction is to
to refresh ourselves as to what we know
about this project and then uh
reach out to try to offer as much
assistance as we can I'm very very
pleased to say that the entire Rec
leaderboard does not have our name on it
today that that will change but you know
as of right now but but this is a
process that we take very seriously
because uh it it can happen there are no
guarantees in audits we we always do our
very best but I think to stick with the
customer when they're in a bad time as
well as when they're in a good time is
the most important thing sorry Jenna
just give us like an insight into what
that process looks like like just one or
two actually the the process of choosing
whether to help the customer or not in
in one of those situations so the the
helping the customer I mean I let me be
clear because I made this same mistake
in Stanford I don't mean free
uh necessarily right like we we will
help obviously for nothing there are
certain things that we can do as
Auditors and and as experts in in the
field where if there's more services
required if it's fixed reviews if
there's other things but the first thing
we want to do is reach out and say hey
we saw that this happened do you need
any do you know do you want to talk to
us about stuff that you're that you're
doing right now I think you you said
that pretty well
can we agree that audits is probably the
the one of the worst terms to describe
what we actually do
I feel so bad using the word auditing or
audit every time I do but these are
really time boxed security assessments
let's just all agree on this as John was
saying these things are not bulletproof
right
um bugs will be missed and we allocate a
certain amount of time to do our best to
find these uh vulnerabilities in your
code but guess what if you give us
double the amount of time chances are
we'll probably find more bugs
um so let's just you know make sure that
we're all clear on what we do up here in
terms of Security reviews and security
assessments okay
and by the way that's that's not just a
cya
it's not
you you said the word partnership or you
said the word partnership
um people wait a long time for audits
it's a very important part of the
process of going live with with a
product especially a product where a lot
of money you know if it goes bad a lot
of money can go out the door uh
customers have their own priorities
customers have budgets customers have
realities and usually it ends up that
we're not negotiating actually about
money but about time we're not
negotiating about money but about the
number of Engineers who can be looking
at it so while we can't guarantee that a
three-week two engineer audit will find
all the bugs we can guarantee that a
three week two engineer audit will find
all the bugs that can be found by three
Engineers by two Engineers working for
three weeks and that that's a different
sort of guarantee yeah okay
um so I think I think Medi set this up
well in that uh by saying like like just
the Magics of what this is what we're
doing how is it defined
um so we're saying time box Security
reviews now
do we need do we need a set of Standards
to make it clear whatever word we choose
do we need to have a set of standards
for this and I'll give it to Chandra to
to
take a crack at it so standards for
Auditors you mean or for yeah I mean one
that's a good place to take it as well
if you like I was thinking more for what
an audit is for what what needs to
happen during the course of this
Security review
well um so you know I I can perhaps
speak from a more automated auditing
point of view but then I think some of
the ideas probably also apply to
human-based auditing uh I think
the hardest thing is to come up with
what with the specification right like
even when you're auditing like what are
you actually looking for I think that's
uh one of the main questions that you
should very carefully think about right
uh and you know I think one of the
things that's kind of uh difficult for
auditing is uh it you it's hard to be
very precise and if you're not precise
then you kind of think that you did a
proof or like you have some kind of
guarantee that this bug isn't there but
I think you know unless it's very easy
to uh feel like you you know audited
really well if you don't have the
specification correctly in your head and
um perhaps a related uh thing is uh
in formal verification there's this
notion of trusted Computing base and I
think that idea also uh you know maps to
auditing like what part of the code did
you audit what parts of the code did you
not audit right and you know you can
only say confidently well with some
confidence that the code that you did
audit maybe doesn't have some
vulnerabilities but the parts that you
didn't you can't say anything about them
right so it's very important to be very
explicit about what the TCB is and what
you verified and what you checked or
whatever you audited
anyone else
I would say
I'm a little frightened of the word
standard in what we do because
I think sometimes it comes from the
place of people hoping
that
we're going to simplify down to some
very simple set of rules what needs to
be covered and then anyone could cover
it and so I do think we need common
language we need in many just helped us
with some of our common language and
understanding
um so that we're working together but I
think what's important is
um if you're if you're a team and you're
engaging auditors
um
it as opposed to you want to have common
language about what will be done what
what won't be done
but this is people also and people with
experience so I think
um getting to know
who are the actual Auditors and not just
I think we were talking about this
before not just the team or the
you know the label but who are the
specific people what is their experience
because those are the people you're
going to be working with and and what's
more important probably than standards
is
agreement with who who's going to
provide you the security review and who
are those people and what are their
skills and what are their ex what's
their experience yeah so I'm gonna like
that I'm gonna go to the other side of
that the interpretation of the question
of standards and well I'm re I'm going
to just steal the phrasing uh from
of Rickard who I was talking to the
other day
um he said who odds the Auditors and
and so you know as a project and I
experience this now
as somebody who's audited before and
understands the industry far better than
most
still scary like talking to like trying
to evaluate
the auditor like for the code that I
have that I want looked at does this is
this the right audit firm and do they
have the right people available when I
it's there's so many moving Parts it's
crazy
um but back to that who are as the
Auditors like is there some way that we
can that we can or should be
holding I don't want to go back to like
holding otters accountable and the skin
in the game thing that we can go around
round about but yeah like how can how
can we help
non-honors evaluate the Otters that
they're working with should there be
some kind of
third-party Watchdog that's ranking you
all or what like there's a I think it's
a place like this a good plug here would
be like to talk about each trust and I
think we all participated in the
standard in one form or another and I
think it's like a dangerous proposition
because what the the proposition of each
trust is that there's levels of how
secure your code bases there's like kind
of a risk assessment of how your code
base will look like after you have done
certain things right so so this this uh
this is part of the eea the Enterprise
ethereum Alliance and I think it was uh
Tomlin who started it I think it is a
dangerous proposition but it might
actually be helpful if you are not super
embedded into the security industry
right again it has
it is useful but it is it should be
taken with a grain of salt because like
if you take level three or like formal
verification as I'm gonna say it that
will probably be a disservice to you so
I'm mixed based on getting standards for
what a good security practice looks like
all right this is a scary one so
oh but I'll go ahead I actually think in
terms of security work they're and and I
think about other niches of Technology
the work that
let's say these teams here do
is very public
it's very public
so in some sense we're watched yeah very
closely
and so this idea that so our reputations
are on the line all the time all the
time and honestly I know all of us up
here we lose sleep over it right like
and and I will say too that like it
opens up when we work and opens up on
contracts and we've had issues and you
know so we're our reputations are very
public and we're all glad we're not on
the rec leaderboard and um so we can
talk about I think it's perfectly
healthy to have conversations about how
could we provide more transparency and
more info but I think we are actually
very yeah well watched right now okay
yeah now now there still is the thing
about what's the right match for you as
a project and I think that's an
important consideration yeah yeah and I
like how you said like you lose sleep
over it I think you know everybody in
this room loses sleep about either the
code that they're running or the code
that they're auditing so
should all go easier on ourselves um so
I just want like I think it'd be fun to
get some questions from the audience uh
so if you have a question raise your
hand uh and meanwhile while the mics are
getting handed out we're gonna try
something overrated underrated lightning
round
no people no firms these are Concepts uh
okay so where should we start uh
Nick
uh plain English specs overrated or
underrated
uh underrated twice it it I was actually
Jocelyn Feist is here is the head of our
blockchain and he was saying you know
you can pick the tool you can you can
pick how it is that you want if if
you're not able to put down in English
what what are the invariants what are
the things that you're trying to
accomplish here you're you know once you
do that you're 75 or 80 of the way
through
it also it just speaks to a number of
things not just for Security review but
also just for your own understanding
yeah to be able to declare this is what
this thing does you can't put a value on
that yeah God
uh Gonzalo
overrated underrated contract upgrade
ability
so overrated
okay
um
expanding upgradability is a bug not
always we went through this so there's a
there's a longer conversation about this
that happened at defy security Summit I
think that is available on YouTube so I
could rent on and on about this there's
also a bunch of articles on the on the
internet about this but generally
speaking it you are just injecting
another
um attack Factory in your code base okay
all right uh Chandra uh how do you feel
about sorry overrated or underrated
reading the code very carefully
truly underrated I think we should read
code extreme we can't just write rules
and you can't just write rules no you
definitely need to understand the code
before you start writing your rules
formal verification is not a silver
bullet it's definitely not a silver
bullet but really but really like how
you know like it's
uh how do you know what kind of
guarantees you're getting from formal
verification
uh I mean yeah I think this is a very
good question right like you
I this kind of goes back to what I was
saying before you know it's important to
think carefully about what you're
approving right and your spec is really
all you have and uh you know when you
say that okay we have a formal guarantee
you should be very careful when you make
these kinds of claims and I think it's
very dangerous to make these claims
because you know
um the spec could itself be wrong and
even if the spec is reasonable I mean
there could be a bug in the formal
verification tool and like then that's
you know that's also bad and uh you know
just because there is a specific bug
that you verify verify your code against
does not mean that there's uh you know
it's just bulletproof and you can just
trust it with your life right so
um yeah okay
um many yeah Europe
um have I asked everybody something okay
all right so you're after many you'll
get something don't worry
um
Maddie
alternate virtual machine designs so not
the evm
overrated or underage
override it
okay
I don't know do you wish to qualify that
or yeah I don't know I've been spending
a lot of time in on the evm so yeah
getting familiar and you know it's got
its quirks.com syndrome probably yeah
probably yeah
um look now we've we've we've definitely
done a bunch of security assessments
targeting different execution
environments don't get me wrong
um but obviously most developers are
operating with solidity Viper at the evm
level it's got its quirks but it it's
kind of working I'd be excited to see
the upcoming upgrades uh particularly
eof AVM object files that might change a
lot of things with regards to security
guarantees or assumptions that we make
with that with the evm but yeah I don't
know I quite like it by now all right uh
Jonathan um
how do you feel about like insurance
against smart contract vulnerabilities
over 800 that's where to start
I guess I'd I mean if I
probably say
um
I'm going to say underrated because I I
don't really think it's
widely adopted and used at this point
and I think it's a it's a uh
it's based on where we're at
as as a you know our risk tolerance is
really high
and our you know our skin in the game is
really high and
um and and our risk levels are not well
understood so it's hard to get insurance
yeah if not impossible for a lot of us
so I think that'll change that's gonna
change for sure and so that's why I say
underrated I think it'll change and then
yeah there'll be Hedges and insurance
and I mean it is happening it's not like
it isn't happening but I think it'll it
will happen more
um and hopefully you know some of the
work that some of us do up here will
help
everybody understand the risk better so
that insurance will be obtainable and
things like that for those who want it
yeah I do I do like I'll I will give one
I haven't given many opinions I don't
think but I think that having somebody
try to price risk
uh who's not an auditor Auditors don't
want to price risk they like look for
bugs right but and it's a totally
different thing to be in uh what is an
actuary
but there should be actuaries and maybe
they are the ones maybe they audit the
Auditors they have an incentive to use
and we're working what I'm what I'm
hoping and and we're doing some work
with with the actuarials and and with
the insurance companies to make sure
that insurance in this industry does not
go the route of insurance in quote cyber
security where I mean I thought it was
quite hilarious about a week ago when
Lloyds of London got hit with with a
cyber and like they can't set Actuarial
terms if they can't even secure their
own systems yeah what we're really
hoping for is that that
people doing Security reviews can set
those standards so that that the
insurance companies are actually as
passionate about what they do that
they're getting into it not just because
oh look at that it's a new adjacent
Market that we can get into and much
more around wow this is really
Innovative stuff we should get in there
to make it safer and we're working with
several people like that and I'm very
hopeful so I'm with you about underrated
interesting okay uh is the audience
ready does anybody have a mic and a
question yeah I do
all right let's just
uh Mike if you I can hear you so I'll
repeat it
oh somebody's somebody's set up okay go
ahead please hey uh so I was wondering
because of all the monitoring tools that
are coming out like photo for example
and there is also private mempools is it
really overrated
oh so you keep I like you're using the
format
uh
I don't know who should I give that to
you later what's the question uh is is
is about monitoring
um is monitoring like mempool monitoring
overrated or underrated
specifically mempool monitoring I think
or just is that is your question
monitoring in general is monitoring over
yeah yeah so uh like for for example
like uh there are products out there
that claim that they could like maybe
detect hacks and all that uh or maybe
secure your smart contracts that kind of
a thing but a hacker could use like a
private mempool to like really send
these transactions across and to even to
react right you your transaction maybe
I'm gonna say
let's talk after
not sure but what I'll say is that um I
don't think there's anybody claiming
that we've solved monitoring in the
ecosystem nobody's claiming that or that
we can
stop threats but I I what I will say is
that
I think we're all I think there's a lot
of agreement
that
security is not just a time box Security
review
if you if that's all you do good luck
and and there are things you should be
doing Upstream
and considering like formal verification
and fuzzing and various techniques time
box Security review part is part of it
and there's things you should do so
pre-release
pre-deployment post deployment there's
things you should do in monitoring is
part of it and so there's value
to an end-to-end security and I think
that's where we're at okay we're
starting to appreciate that and I want
to seek in quickly while we move the mic
over uh
go ahead Alex you're up next Mike don't
worry hey uh do your clients request
public reports and when you write
reports who do you write the report for
um yes so I was I was telling one of my
team members the other day that uh what
we sell effectively is PDFs obviously
joking a bit but yeah uh
um yes it's critical the output of the
work that we do is a security assessment
report which you know contains your
standard executive summary detailed
description of all the vulnerabilities
identified and recommendations
who do we write that for that's an
excellent question we write that for our
clients now the problem that we may see
quite often in our space is uh you know
readers
um sort of going through well users of
the protocols that we review going
through our reports and not necessarily
understanding again the concept of a
timebox security assessment in a lot of
cases the scope that we actually
um go through is a limited part of the
entire protocol these happens quite
often particularly with very very
complex systems
um and unfortunately some users may use
our security assessment reports and say
look this protocol is absolutely safe
because this auditing firm has only
found a couple of you know mediums and
lows and informationals and they've all
been resolved
it right exactly that's right they
passed the audit big check mark none of
us here provide you know any big green
check mark in our auditing reports
there's plenty of other firms that do
that unfortunately
um so yeah I think I think you're
absolutely right the audience of these
these security assessment report is um
is a tricky one
um
let me sneak another question if a
client asks you to remove something so
or rephrase something are you writing it
for a client yeah this happens quite
often I'm sure I'm sure that like it
happens quite often yeah I'll take it uh
no no no no I mean I mean a client can
argue on the severity of a bug right and
that's absolutely fine you know we're
more than happy to have that
conversation and be like okay why do you
think this is not a critical like look
I've got
systems of chain in place to prevent
this from happening that I would
actually drop down the likelihood of
exploitation I'm like okay fair I'm
happy to drop down the likelihood which
would effectively impact the severity of
the bug but removing a bug from a report
if it's valid hell no
now this goes against the trusted
advisor role which we play and and how
to read our reports or I think all of
our reports is that it isn't that check
mark and what I want to see from a
customer is I want to see them come out
with our original report which we will
not change and so you as as the public
can look at the report and see the
problems that they faced and then you
can look at the fix review report which
is separate or an appendix it's just
it's it's different and see okay you can
actually watch the progression as
they're taking the advice they're
understanding not just how to fix that
bug but how to fix bugs like that from
happening at all and and some companies
are afraid to do that because people
will think that they're not good at
security I look at it as the exact
opposite a company that is proud to say
look these are the challenges we face
and here are the innovative solutions
that we came up with to address them
that's where I want to look because now
they're they're actually taking our
advice and they're and they're doing
what they should do with an office is
there I want to like is Chandra is there
are a difference
in the way that like a report needs to
be presented when it's like heavy on
formal verification
I mean I
think we
similar reports and like Maddie said
right like we have the severity of the
bugs and we explain exactly what
property we wrote and why that
corresponds to the bug and
I think the only additional thing that
we very much emphasize is that you know
uh like this like
be careful that don't just take this as
like the you know uh this is not a
silver bullet and like the yeah I think
there's no difference to be honest in
terms of the reports we write
um okay Mike thank you
um so uh disclosure I work with Jonathan
and open Zeppelin but I didn't tell
about the question ahead of time so
there's no front running here
um if you could wave your hand and every
single developer in the space but most
especially your clients just
automatically follow the best practice
without you having to tell them ever
again
what would it be yeah yeah it was a bit
echoey but uh if you could magically
like if you could be sure that every
client if there's one best practice that
you wish every client would would adhere
to what would it be
um
who's feel yeah all right I've got one
do not change commits meet the reviews
please thank you
should I should I start formal
verification at the very beginning and
continuously throughout the project or
only at the very end oh that's such a
good question
uh so the question is do you want should
we start formal verification towards the
end or do towards the beginning of your
development
so this is a very interesting question
you should start as early as you can
that's uh based on you know our
experience I think that's the right
place to start thinking about uh formal
verification I think the most important
thing there is you know you uh it really
helps you sort of write your code in the
most simple modular way because that's
really what is best suited for formal
verification and you know it just it
automatically just prevents you from
making certain mistakes and I think it's
extremely important even if you are you
know no matter what kind of verification
technique you're using the earlier you
start the better and along the same
lines you know writing your specs out in
English and also like you know gradually
making them somewhat more formal I think
the earlier you do it the better like
once you've already implemented your
code try to come up with an invariant or
like trying to write down the pre and
post conditions and stuff it's it gets
really hard so I think there's like I
like the shift left security meme which
is like it basically just the idea if
you think of your time and your process
is moving from left to right writing
writing your code then writing your
specs and tests is doing it on the right
hand side so the more that you can do
any of these things your specs formal
verification test writing
earlier in the process generally the
better your outcome will be absolutely
yeah yeah I think yeah so I have a
question for you I think Jonathan and
Nick were touching upon it but crypto is
a very narrative driven industry in
general and if we zoom out a little bit
and get outside of the security Eco
Chambers what are some narratives that
you want to see and what are some
narratives you want to correct that are
floating out right now in terms of
security did you catch uh I'll try to so
you're saying you said it was this is a
very we're in an early stage right now
we're in an echo chamber of security
conversation and we need like
I'm struggling yeah I was going to say
if we make an assumption that security
Community within crypto uh is is on the
on the table and there is a broader
developer Community outside as well uh
who are all developing building the
tools who are uh susceptible to certain
narratives what are some narratives that
are floating around around secured
around how people see security audits
around how people see Security in
general that you want to take the chance
to like correct if you see anything
wrong what does some ways
all right so I'll just repeat it for for
everyone from what I heard and correct
me if I mishad
um bad narratives in our industry that
should be corrected hopefully quickly
and early on
um is anybody
got done what are you angry about
see I know I know how to like
um I'm angry about a a lot of the raises
that happen in the security industry
recently I think think the the the the
money and capitalism is creating
legitimacy
for for people that have not
demonstrated proficiency so this is a
narrative that really grinds my gears
for example
um wow and uh yeah
yeah I'll give someone else I'll sneak
in a smart contract over availability
here all right since Gonzalo started
with things that make us angry I'll do
one too that um well I think that the
our desire for cost efficiency
and time efficiency is not in our best
interests and I mean gas savings and
speed of transfer from a layer two to a
layer one through the bridge that was
just developed yesterday like and our
lack of acceptance on latency on
anything so that we could do more
security verification so I think these
things work against us and I understand
why we're doing them and how they're
helping initially but I hope the
narrative changes so that we
um aren't as worried about gas savings
and we're more and we aren't as worried
about
speed and we're a little more worried
about security and safety for the users
hello everyone my question is is uh if
auditing is good enough
or would you consider like uh
Alternatives like open arena or Bounty
bolts like hats Finance for example
or maybe deploying two medium tier uh
Auditors or just one top tier auditor
per contract considering timeline
so I'm not sure I fully understood the
question but I think it's
please correct me if I'm wrong but I
think you asked if auditing is good
enough then why do we need uh the
bounties let's make it more General I
like we sort of I I think I think I'll
get we will get this is like like just
generally we have some new alternatives
to audits that have emerged giant
bounties uh and and like code Arenas and
Sherlock doing these uh like auditing
competitions that are a little bit like
a Time boxed uh Bounty program so yeah
um how does that uh how does that like
substitute or compliment auditing I
don't know this I don't know if anyone's
like we might yeah yeah but my answer is
is back to what I said a little bit
before which is a bounty can give you
that this bug which is interesting but
it's not really as interesting as as how
you are working with this is a this is a
holistic view of of all of The Code by
by Engineers who are deeply and
intimately involved with the code to the
same extent that that your internal
people are but they're seeing it from
through a different lens they're seeing
it through the lens of people who have
literally watched everything that could
go wrong go wrong and and being able to
see
mistakes in everything from
documentation to to just your overall
maturity it's a different set of things
if you want if you want to find specific
bugs and fix those that's great there's
other things that are that are outside
the the general scope of what we do in a
Security review uh I'm not suggesting
that that audit or Security review is
the the be-all and end-all but it's also
uh different from a lot of things that
I'm seeing out there and as far as
automation goes we use automation to
find sort of low-hanging fruit or to
point us in the right direction so that
humans can look at it and make educated
decisions about what we think and
there's a lot more than just run this
tool see what you get and hand over the
report anyone else quickly we're running
low on time
um sorry sorry sorry no no that's I
meant on the stage like if anybody
wanted to all right I'm gonna be selfish
too there's something in the time we
have I really want to plant this idea as
much as possible is that it's not the
auditor's job they're not selling you
Security in my opinion they're selling
you like insight into your ability to
write secure code
so you need to take responsibility for
writing secure code uh and not act as if
like you just get to write code that
does cool stuff and then give it to the
auditor and they will make it safe for
you so you need to focus on being able
to write secure code whatever process
you think that might need whatever tool
you think that requires and then you
give the other your code so that you can
understand how close you are to being
able to write secure code amen
it took me a long time to get there so
it's a bit of an epiphany and I'm trying
to unfortunately we see what John was
describing way too often and yeah it'd
be great if we can all collectively
shift
um towards a different
um yeah definition of what our work here
means
um yeah thank you thank you okay there
was another question so let's have it I
think so um Foundry when Foundry first
came out uh I was surprised by the fact
that it included fuzzing so my question
is when does fuzzing make sense on these
code bases which are generally open
source and small and how do you approach
fuzzing in your use for auditing
fuzzing makes a lot of sense all the
time it's we're going to like the shift
left the shift security left narrative
and all that like you should use all the
tools literally uh you you should write
specs if you have specs fuzzing will
work better right
um so I I don't yeah you should just do
it all the time use Foundry and use all
the other tools use a kidney use Harvey
by the way I forgot to say I've been
announcing this the entire week but this
is the panel we at diligence will be
open sourcing all our tools just like
trailer bits we are a bit behind
schedule but at least very late than
never so even Harvey yeah yeah yeah all
the code bases so yeah
foreign
did you uh I just well yeah I guess I to
add to what Gonzalo said I think uh
using tools is actually good so fuzzing
makes sense to me I mean you know again
it's not a guarantee but you know I
think uh it's just like good software
engineering right like you write code
and you want to write tests and you want
to do fuzzing and you want to write
specs and these are all good software
engineering practices and I think they
also just are they carry to this domain
as well so it makes sense
um the one the one comment about that
you just made well both of you like uh
that the Declaration of things the
ability to set forth the spec uh and
then and then fuzz around that that's
really important that first part is what
Jocelyn and and Gustavo are going to be
doing tomorrow there's going to be a
two-hour workshop on how do you declare
these things how do you actually set out
and and describe in variants and how how
can you use those to make your fuzzing
more effective so I would really
recommend that that you go to that so
just to be fair I know like we're at a
time by this timer let's give every
speaker like 30 seconds to sorry
all right so we started the questions
way early we've got tons of time
um I should have read the manual
um did
uh Brian I know Brian had his hand up at
one point
um and
I'll get you please just get just like
I'll I'll repeat it through the mic so
go ahead
that's that's cool is there a way to
make like the the output of of audits
machine readable
um and it's interesting because we we do
actually
that's really interesting and I think
we're gonna take that up and talk to us
hit us up afterwards
um I I think it's gonna really depend on
the writing style of of the security
firm that you engage with
um but it is something I'd like to think
about a bit more yeah
um
perhaps we could parse some of those
reports into a readable format or a
machine readable format that shouldn't
be too hard to do we write a report in
reports in latex so
it should be pretty trivial for us to do
cool so uh so try actually we do have we
have been working on a related things so
um I don't know if you've ever heard
like there's things like deoxygen or the
oxygen I don't know how to pronounce it
and like there's also uh Nat spec is
another similar Library so uh the nice
thing about you know if you're if you
have a formal spec it's actually a
little bit easier to like sort of
convert that to a format because you can
annotate your code and then you can
automatically generate some kind of a
formatted report from from that so I
think it's a good idea
uh Hey guys Kevin over here uh Hey
friends
um so having come from sort of the
security side and now being protocol
engineer and founder
um I'm wondering about this this line
you tread as an author of being very
slow methodical you know heavy testing
core protocol must be super secure to
now okay now I'm wanting to move a bit
faster maybe their integration contracts
or something that's not in my core
protocol
um you know how much time am I spending
and testing and auditing how much money
am I spending there versus move fast and
see if things work I wonder if there's I
have my own opinion but interested to
hear what's on the stage I'll just
quickly I'll say um the most secure code
is the code that never hits mainnet so
just keep testing
I also think it depends on how much
money you raised or how much money you
have and now it's like it's very it's a
very practical thing you know like you
can spend a lot of time and resources
doing doing writing tests but at the end
of the day if you run out of money and
you only have the test you're probably
okay cool it might come off as somewhat
pedantic but I'm curious looking at the
different way that auditing firms do
severity or impact and then difficulty
slash likelihood I'll be honest I like
when
I prefer likelihood so that it's like uh
difficulty is low when likelihood is
high so I was just wondering you know
High likelihood and high impact would
equal High severity have you seen
clients get confused about this
different and without being overly like
strict as the industry is still
developing I'm also curious about like
how we might classify things trail of
bits has a nice classification
swc seems like outdated and not updated
as well as or maybe it could be going
forward are there things that we can do
as an industry to improve and start to
I hate to say solidify but structure our
information architecture a little bit
more so that clients can understand when
they're getting multiple audit reports
and reduce confusion I think I think
it's uh yeah it's a great Point as you
said uh well for us for us at CP a high
likelihood high impact critical
um and I think we all use perhaps
similar types of risk rating matrices
um it it's probably beneficial if you
could standardize on on how we would
write severity so as you said it's
typically likelihood of exploitation
impact of exploitation but there's
different levels so we only have we got
four of them at CP so informational low
medium high and critical that's actually
five
um but yeah I think I think it's a great
idea there's been a lot of initiatives
over the past few years on The Ether
security channel to try to standardize
some of this stuff it's never happened I
don't I don't really know why
um but I think it would be it would be
quite beneficial
I'm going to give you a bit of a left
field question so there's been a few
regulatory proposals that includes uh
software security best practices in The
Proposal uh what sorts of standards that
are driven by regulations could be do
you think could be damaging to the space
and what ones do you think could be good
like so so almost like uh yeah the
typical like like a regular comes out of
nowhere and they're maybe a bit naive
what kind of damage could they perhaps
do by uh
misunderstanding well I think I I don't
know if it goes to like
the intent of your question having to do
with
structure of code I think we all know
we're concerned about
regulations that would impose
requirements on protocols
to be able to block certain parties as
an example so I think the biggest
a lot of our concerns are around
regulations that would have a
requirement so I don't know if that goes
to your thought about the question
having to do with
specific
processes
your contracts must be upgradable for
whatever reason
those things are always damaging if
you're talking about regulations at the
code level I think there's no way they
can be good you know like I I think that
um as an example having worked in
uh
security before and in the in the cloud
space
the sock 2 standard
and actually I believe it has a lot of
really good aspects to the software
development the secure security focused
processes of your software development
life cycle and I think those are good
practices and if
teams were transparent about the
processes they follow I think that's
healthy and if held to a certain
standard all those things always have
downsides in terms of the effort that
goes into auditing them and proving them
but yeah the life cycle of Regulation is
really tough
um getting getting regulators and
government officials to understand the
key areas
they have a goal of what they want to
accomplish but usually they're they're
rather far behind the technology I think
it's up to us everybody here we
certainly do this I bet everybody else
up here does this we spend a lot of time
talking to Regulators about what is
possible and what isn't we spent a lot
of time talking about the intent and
trying to come up with sort of more more
open suggestions to Industry as opposed
to specific because if if you say thou
shalt do the following it's always going
to be out of date by the time it passes
and and it just becomes very very
difficult so I think that the best thing
that we can do the most practical thing
we can do is educate those those people
who would regulate us thanks for
clarifying Brian uh yeah that makes a
lot more sense
um just to give you guys maybe an exotic
perspective I was catching up with some
of the Regulators in Australia a few
weeks ago and this is exactly what they
were asking for they said what should we
be you know requesting from projects to
do in terms of security practices in
terms of security development secure
development
and you know what should we be
regulating in the space so I try to sort
of convince them to go down the
educational part and um and yeah this is
this is spot on I think it's certainly
something that is happening at least in
Australia these people are starting
these conversations and are starting to
realize the need of you you can't
enforce these things on projects but at
least you can have these set of
guidelines and point your users your you
know citizens to these guidelines and
get them to demand these standards these
set of minimum requirements from the
projects they interact with
thank you
[Applause]