so my name is Martin holes Linda and I
work with the security for the etherium
Foundation and I also work with a gift
client and I'm going to talk today about
basically things that have happened
during the last three years so I became
the security guy in basically when I was
Def Con in Shanghai and we're now in all
second I'm going to talk about the base
layer the infrastructure of ethereum so
what is the theorem infrastructure and
[Music]
the building blocks of ethereum so
etherium is a blockchain and one of the
things that can be attacked and it's a
vulnerability which can be is the block
processing so a client gets blocks and
the block consists of headers and
transactions and some other things
and they need to be validated according
to a very particular set of rules it
can't be too much validated and it can't
be too little validated so there's very
intricate rules and it's extremely
consensus critical because if clients do
it differently we have an immediate
chain split there's this EVM which I
think is the one thing that everyone
knows exists in ethereum and this is
where the smart contracts operate and
the smart contracts execute on top of
this thing called the state which is
this global state of where everything is
right now and basically they accumulated
result of all the transactions in the
history of ethereum also highly
consensus critical and but aside from
that they can also be denial of service
attacks that causes slow blocks and this
is what happened during the so-called
Shanghai attacks we have networking
protocols one of them is the eath
protocol it's not actually a networking
protocol it's it's the ethereum specific
application
recall so that two nodes can communicate
over each other and one can say I need
some blocks and they can do
announcements of new books etc and this
protocol contains the building blocks
that clients can use for their syncs
there are neighboring protocols for LS
there's like science and parodies own
custom warp sync this piece is not
consensus critical per se but of course
it's very important that it works
because otherwise we can have problems
getting no sync and there can be denial
of service and there can be lost blocks
and largely orcs and generally bad
functioning of ethereum and the eath
protocol operates on top of the
peer-to-peer protocol the p2p protocol
which sits on top of TCP and it has just
some basic guarantees and identify no
verifies node identities also and not
concerns is critical but it's the
backbone for the upper layer
peer-to-peer protocols so if this were
to fail then of oh yeah you'd have the
same failures as the each protocol if
varium peer-to-peer networking obviously
is built upon the premise that I as a
node can find another node on the
Internet which is not a trivial task but
for that we have the discovery protocol
which currently is in version 4 there is
a version 5 being worked at and this
uses academia distributed hash table to
announce presence and using this DHT we
can find nodes according to a distance
metric and they can hone in on each
other and the distance does has nothing
to do with geographic location or
network distance the discovery protocol
is not based on these p but UDP so it
has no built-in guarantees from the
lower layers about
for spoofing protection it's trivial to
spoof by piece and this protocol can be
successful to a couple different types
of vulnerabilities one of them is
Eclipse attacks where a node is
partitioned off from the rest of the
network and they only wants to
communicate with our nodes controlled by
the attacker
this protocol can also be used to do
distributed denial of service attacks
against third parties that are nothing
to do with ethereum and amplification
attacks which kind of is part of the
DDoS and an amplification attack can be
said to be an attack where I have a
small resource that can be a small
network bandwidth and I could sell my
and bytes out to the network but the
effect that is reached on the victim is
amplified by a factor of something else
so basically this is one way to develop
the attack service surface I omitted
some parts there's also the sync
protocols but there were only four forth
things in this slide template so I had
to divvy it up like this so I'm going
from the bottom up here and during 2018
and 2019 there was actually quite a lot
research into the discovery protocol and
there were two papers sent to us before
they were released to the general public
about the Klipsch attacks and some
people also reported by the bouncy bug
bounty and vulnerabilities amplification
DDoS attacks and Felix and Frank on the
guest team found even more issues and
one of the eclipse attacks that was
found in the first paper is that gif as
a client
had dedicated half the connection
numbers to be incoming and half to be
outgoing however if it hadn't already
failed done all the outgoing connections
it allowed the incoming connections to
fill up all the slots so an attacker
could simply just over we're overwhelmed
I guess no with incoming connection and
have all the connections to the thing
they could also see that there were
other ways to fill out slots by pinging
pinging them and looting the identity
database and the third attack they found
was that if you manage to disturb the
timing at the time that a node has other
peers will reject him and forget about
him and he will reject other peers and
only the malicious payers will accept
him as for amplification attacks here's
one example that Jeff was susceptible to
so if a malicious node spoof and spoof
pinged the target the target would reach
out to the actual victim with the ping
with the poems or in the new ping and
the malicious node could send in the
poem pretending to have seen the first
outgoing ping from the target and he's
supposed to supply reply token but
didn't actually validate the reply token
so at that point target believed that
malicious node resided on IP address 1 2
3 4 and if the militia of them sends a
so-called final request which is only
213 bytes or so it causes a final
response called neighbors to be sent to
the victim which is 1.5 kilobytes
there's an amplification like an
amplification of about 7 on these
attacks which is not trivial more
complex example could be made also where
you set up the ball
ending with two notes and then had one
of them lie about the location of the
first one and the first one then do is
moved find out requests from that IP
address which he'd lied about and ankles
the same effect moving upwards in the
network stack to ethan peer-to-peer la
lair in the beginning of 2017 we saw
that the network traffic was really up
here so much noise and we started
analyzing and looking into what caused
that and found that during about two
hours and note would typically reject
165,000 transactions because they were
invalid so 19 invalid transactions every
second just streaming into the client
problems were that one of the clients
parrot I didn't remember who they'd sent
transactions to and another issue was
that they didn't actually garbage
collect invalid transactions asked state
progressed over blocks
so we've if they were included they were
removed but if they became invalid
because of they projected progressional
blocks and they were not removed so they
were just constantly being recent and
after we looked into that and and nail
out iron out those issues you could see
that the the graphs on these monitoring
notes which we put up started levelling
out and that the incoming traffic on a
node went down by a factor of 20 to 30
during January to March in 2017 and also
the only is this protocol level there is
the sync protocols as I mentioned and
earlier in April this year we saw a new
kind of attack which we hadn't seen
before which was a target attack or the
fasting protocol which was a grieving
attack where as soon as a node joined
the network the attacker who was
scanning the DHT tables a new node would
instantly connect to that node and tell
him
Hey I had the blockchain here I'm full
of sinked at block five and the victim
having no other source of truth would
tell him yeah please send me over the
states on the blocks and that of course
happens very quickly since what it was a
very small chain and the victim would
then be fast synched to block five or so
and after that switch over to block by
block sync effectively doing a full sync
to book one the remaining eight point
five million blocks so it was an
interesting attack because in no way
does it benefit the attacker he needs to
actively run nodes and run some custom
software to scan the DHT and do this
attack and gain absolutely nothing other
than what's the etherion world burn a
bit a bug bounty from junior-m
was that there was a vulnerability in
the protocol and gap where someone queer
state entries with a so-called kind of a
null requests were crossing the Tri
metal route an instantly crash and
appeared so these kinds of bugs if even
an attacker has them he can iteratively
shut down the etherium Network but just
connecting to them mapping the the
amount of the DHT and connected them is
shut them down iteratively which can be
really really bad as for block
processing and in 2018 there was a live
consensus floor that happened on Roxton
Robson is one of the test nets and what
happened was that someone had sent one
of these EEP 86 transactions on Rob
stone
now EP 86 and was Anita proposed a new
transaction format for account
abstraction and it was never actually
roll out it was never actually included
in any hard work but parity had
implemented it and they mistakenly
enabled it in the client and not only
for test nuts
so this was kind of a real critical
issue and we were just waiting for
someone hacked this someone did this
found the floor exploited the test net
and we were just waiting for it to hit
maintenance fortunately it did not
apparently quickly released a fix and we
could see eventually that actually there
was no attacker per se
it was basically the researchers who
were playing about a bit with these kind
of transactions so not all instances
caused by malicious intents and in the
block processing area there was also in
March a very long-lived consensus bug I
think that might have been the most
long-lived consensus bug we had found in
the block processing engine whereby both
gas and parity and did it erroneously so
blocks have time stamps a normal time a
normal block cannot be too far in the
future because the client won't import
it however an orphan block or an Omer
block doesn't have any restrictions on
the time stamp they can be how far much
in the future they want and if they
exceeded the UN 64 yes would just yeah
wrap it around on you in 64 and parity
would cap it so both would be incorrect
and a calculating correct block cache
and it would cause an a consensus
failure with the quirk that none of them
would be correct
this would be kind of hard to exploit
because you would need some mind both
the quark uncle and a block within a
period of 7 blocks and then roll it out
to maintenance so you'd have to have
quite a lot of mining power to actually
do this kind of attack I'm going to have
to skip a bit in the EVM portion most of
you might know some of these so Shanghai
attacks arrived which
mentioned a bit caused a bit of problems
for us during the about the month there
were denial of service attacks because
first of all in optimal suboptimal
client with limitations and secondly
because of gas miss pricings and there
were two hard Forks that followed on
that and there was a naina chain splits
because of how the reversion was
implemented and how that whole change
that happened we know how it happened we
haven't really figured out a good rule
to explain how it happened and put it
into the yellow paper right now the
situation is that all the clients have
particular right and DS special rules so
you can see here a bit of code from the
Goethe areum codebase and get up code
from the aleph codebase and they
basically just oh we fix this a little
bit it's not it's not don't look at it
this is ugly we just know that
fortunately this is this exact situation
will never again arise because the
conditions that would enable this to
rise can never again happen on the
maintenance in January I had to actually
remove this slide because each see was
still vulnerable last Friday in February
2017 that was really interesting a
consensus bug which was a combo
consensus bug that could either be used
to cause an instant chain split or it
could be used to crash all the gif nodes
so the stack requirements were
misconfigured for the swap top and
balance of op codes and the thing is if
the stack requirements are off if it
leads to I'm not talking about the EVM
stack not like an ALICE type if it's an
underflow death would immediately panic
in all the nodes that were processing
this block all the gift notes would
immediately drop off drop off the
network if it that was like overflow
that will lead to a consensus issue
where GAF had a different state route
and parity
and you would split the chain so any
attacker which found this could just
choose which route to go and it would
have been trivial to exploit and trivial
to detect if it'd have been past naively
so what we did was we read refactor the
entire code that that's configured the
EVM of codes and two weeks later we
reversed the fix reversed the the whole
refactoring but left the fix in in 2017
some attackers were hitting the gas jump
test analysis so we saw on our live
monitoring system that some blocks took
like 24 seconds to go through and yeah
so we had to rework the whole jump test
analysis very quickly bring it down a
couple of orders of magnitude in
processing time and there were some
ideas that this was possibly the
Shanghai attacker doing this we don't
know I don't have to have time to go
through the whole byzantium but in
bicenthium when we were approaching that
we really started getting payoff from
passing and it found all these issues
and you can see the last one here
happened on October 25 that's actually
nine days after buy something for kids
and not only that not until February did
we uncover the hopefully last bicenthium
bug
it was Cassidy drew found an issue with
our biometric pairing which was we
couldn't actually fix it until a couple
of months later when we swapped out the
whole VM library for an assembly version
written by CloudFlare some takeaways
from this is that monitoring and metrics
and graphing up things really helps to
see the overall network health and see
what problems there are and what
problems there might be in PTC loggers
giving a talk later about monitoring in
live with her
here's a chart that can display very
detailed what happens when we process
the blocks and exactly where the time is
spent and these kind of things really
help us to find what's going on
similarly for the network traffic's you
can pair with a naive one which I showed
earlier here's a much more detailed one
yeah I mean of course it's really good
tooling to to find out what happened
when something happened and why it
happened and we need good communication
clients between teams to quickly
coordinate to fix stuff when when
goes down on the minute and one thing
that's been really important from day
one and I think it really has been great
is that we have there's no multi culture
where we don't just rely on one client
to hold up the network and I hope that
that node multi-culture will continue on
ethereum that's it for me thank you
[Applause]
[Music]
you