so but before we start I'll talk a
little bit about what we'll go over
today so talk about a little bit about
who we are at white black and what our
role is in eath two challenges to
networking and eath 2 we'll go over a
recap of the Interop lock-in that we had
in September and what our role was there
and a roadmap as well as a Q&A at the
end so white block is a team of
engineers building tools to test
blockchains
servitude systems protocols what have
you we have a core product called
Genesis what Genesis you can fully
automate the creation and deployment of
a blockchain you can configure network
topologies deterministically tests how
they'll react in real world and worst
case conditions we can do things like
implement a network partition to force
forking events and see how your
blockchain will react and reconciliate
that information and the reason why
we're talking about it is because we
will use its functionality to test
interoperability and eath - so here we
have everybody's favorite diagram the
networking old paradigm right so sorry
it's a little messed up we just
downloaded to PowerPoint and didn't have
time to fix this but we so you have a
client currying server for information
that it desires server acts as the
centralized point of truth for that
information right everybody knows this
model that's my world here cool so then
you have blockchain networking which is
very very difficult you have a bunch of
nodes in the network that are tasked
with challenge to communicate over a
large network and communicate with
unknown peers at that so it gets pretty
difficult in eath to specifically this
means that you have nodes receiving
messages of blocks and add to stations
and propagating them to other nodes and
they do this over an asynchronous
communication pattern called gossip sub
and we'll talk a little bit more about
that later
so nodes will receive a message and pass
it on to their interested peers and the
idea what gossips
is that a message will efficiently
propagate throughout the network once
the originating node broadcasts it out
so nodes will request information they
don't have from other nodes so if a node
has adequate resources it learns of a
new block it will RPC request that the
block or the node that has that
information to receive it cool so there
are a lot of challenges in distributed
networking
I don't know how where you are of them
but when we listed very few of them but
we're actively at web block looking at
these challenges along with eath to
client teams if two research teams and
other teams in the space looking at them
so there's the issue of peer discovery
how do you as a node find two other
nodes that you want to talk to in a
large network the issue of distributed
hash tables so dhts
how do you store information about those
peers in the network there's a issue of
security how do you increase tolerance
to certain kinds of attacks that are
prevalent in distributed networks like
eucalyptus attacks what have you the
issue of block and attestation
propagation for which we will use gossip
so again we'll talk a little bit more
about that in a second the issue of loot
nodes in the network which are points of
centralization perceived to be points of
weakness issue of cryptographic
verification checking to make sure that
signatures are valid hashes match block
information and then the issue of
syncing so how do you how do you
efficiently sync and securely sync
information against other nodes in the
network so I'm sure that you guys have
already had like your ears talked off
about it would be 2p at this conference
it's been like a super hot topic at Def
Con and if you want to go we're only
gonna cover it briefly but if you want
to go like more in depth about it roll
Kripalani from the p2p did it talk about
gossip sub like two days ago
I'm sure the slides are already up on
Twitter somewhere and the video will be
uploaded but he did a really good deep
dive into gossip 7 but we will we will
cover just like very basically what
gossip sub does so that we have a
to start off at so gossip sub is an
epidemic broadcast protocol it's an
implementation of a pub/sub
communication pattern and it's designed
in such a way that information will
efficiently propagate across the network
so nodes will listen for information
that they want and pass it on to their
peers that want that information pattern
will continue until all nodes that are
interested in that information have
received it and in the network so gossip
sub is an efficient protocol for
propagating blocks and at the stations
in eath too which is why it's used as
the eath to networking stack so back in
april we actually did a few preliminary
tests on the PDP and they were not so
great but again like role mentioned that
flip p2p is in alpha mode right now so
there are a lot of parts of it that
still need to be like massaged and and
improved to make sure that it's going to
be ready for Phase II our launch
so we at white block will actually
continue testing and auditing the p2p to
ensure that it will be ready for a
successful phaser and I'm gonna talk
about that
thanks for me so why are we testing us
observe right so the point of this is to
really assess the real-world performance
of gossip sub under very adverse
conditions dropping gears bad bandwidth
packet loss huge latency between the
nodes knowing that in the case of if -
we have 6 seconds per block so if you
want to really gossip everything you
would want to have 3 seconds to go sip
the whole information between all the
peers so they can decide and start
voting correctly on the new block one
thing that we can reach week during that
testing is a different network
topologies so if you went to Randall's
talk he said that you have a minimum of
4 peers a maximum of 12 is that really
what we want to go for should we have
more peers we have a few better peers
should we start working a little bit on
some of the working wants to do for
episode where we're going to have
optimizations where we're going to
prefer nodes which are closer to us
which have lower latency which are more
responsive we're going to look into that
so to get to work first thing we need is
when we did our past name Pro we used to
go to p2p demon this time we're starting
from scratch with a company called the
agency Enterprise and they're really
great they work very closely with
protocol labs so Adam Hannah has been
building a leap PGP client which is
based on the Goldie p2p core library and
it's a very simple client that just is
going to output a bunch of metadata
information about the message right when
was it sent who sent it who is it for
how many hops DC for that message
you know how long has it been around
what type of like how long was the hop
between those different peers we also
hired an academic on our team who is now
doing a literature review to look at the
different topologies everything should
be done there namely right now I'd like
to try also to do some discovery
you know associations so we can do
academia approach to just let the
network self organize but it might
actually not be the best way to organize
appears so we're testing nozzles at
different aspects so talking about
network topologies we're going to strain
it or gain credit sequence of peers
we're going to create a mesh of tears
just like redundant hubs between each
other so we're going to really see that
you know from node 1 to 100 we're going
to you have to traverse all the other
nodes to get to that one so this is in
progress you can check out those two
repos the first one here is our project
management repo this is all the issues
kind of reflect the work from the team
you can take a look
contributions are very welcome your
feedback in issues you see the work
itself the benchmark tool is actually
under agency enterprise it can take a
look at it you can run it right now the
idea is that we're going to talk right
at work and then deploy it on Genesis
we're getting around you know 3100 of
those in parallel and we had some
unexpected good news and of familiar but
we got access to an academic
facility to run all those tests
something called amil abhinaya so we're
starting testing actually this week you
got a guy on this we're going to publish
all the raw data sets for every test
suite on Google Cloud Storage you can
download those there you know one point
five gigabytes each so just quite a bit
of data and then we'll also share the
Python parsing script that we use to
read all the logs to make sense of them
to compare all the statistics the median
the all these data in will make that
open source as well probably and there's
a the PM repo so the idea is that we
want to be completely transparent make
sure that our work is being reviewed by
the community and if you have an idea or
a suggestion for a different type of
testing you can come to us you can ask
us questions we can have a conversation
about this and we can come together and
collaborate on all those things so to
switch gears this is coming a little bit
also from a workforce if you're m2 so
just going to do a little recap on the
interrupt workshop that we did in
September here you can see these are
stand up in the morning have Danny Ryan
F telling us how things are going we
were going by team here you have like
different teams working together on
working on the same laptops so the
purpose of interrupt for us was to
establish communications between the
teams to make sure that they had a good
process to work together coming in to
interrupt we just wanted people to just
take one laptop to clients make them
work and see he was able to even just
gossip a blog say hello
have some information we had some great
tooling from pato lambda from the Fenian
foundation so he gave us a way to
regenerate genesis states like 30
seconds or 1 minute in the future that
everybody could ingest
as well as the list of validators that
would be associated with each client and
then would give us the ability to very
quickly set up this type of scenarios
you know initially what the clients did
was only one client in the test net with
a validators everybody else would be
passive and just ingesting old data and
as we got closer to the end of the week
everybody was able to
very same time so everybody was
processing blocks we had to get
everybody going a good caveat of this is
that solo jury because we had to kind of
go with the flow of those different
clients some cons did not support sync
for example so if they studied after to
Genesis time they would not have a good
time working with the others right so
you also to give you an idea it was very
manual they would have a Tmax
kind of window at first start first
client look at the inno dove that client
paste that into the start script of the
second client start the second client
start a circle i n't cetera et cetera
very manual but you know for first try
that was that was a good try right so we
had a few panics we had a few
interesting crashes we saw all the
things that could go wrong and well
eventually they managed to get some
clients on laptop which is a pretty good
result much better than we anticipated
going in so yeah eventually what we want
to do now is to actually productize this
so recap of the success we had some
clients talking to each other on a local
machine a little harder for us we're
using docker and now we want to
publicize what they've done into weather
is completely automated so for us it was
a little harder because each client is
ever so slightly different I'm gonna go
into that the way we run things with
Genesis is that we pretty much take
whatever docker image are blocked in
client has and we just automate its
creation in its lifecycle as part of a
platform so it's very easy for us once
everything is in place to go very
quickly to a destination to just
generate many of them so things that
kind of took a bit of time to figure out
we saw that you know if even so
everybody is using the p2p when you have
an address for a client for a node in
Limpy GP can do a couple different
things you can have the public key of
your client being published to all your
peers or you can choose not to do that
and when they connect to each other
that's when the public key get exchanged
right so in that case for example
sometimes
just barf because didn't want to have
the public key of the other node in
there so we had to start tweaking for
that like night-hunting like that or
should we have to change the start
scrape just for them etc we saw also
that they have different ways of storing
the identity key the key that actually
represents the identity of the note lead
PGP by default is going to use a
portable format which adds two bytes to
the beginning of the the string that
would store some clients like that as a
exactly similar string being passed some
like that as a basic ste for string to
be passed and something like that into a
file so we had to account for all those
different variances of the way they
would consume the information all this
of course works very well when you're
doing static peering at this point I
think we have two clients that do this
v5 it might be wrong so that's something
that we need to look at next so that
we're going to talk a little bit about
that quickly for us it was kind of a
great learning experiment working to
generalize or framework to make it much
easier to recognize those those clients
without changes as much as possible
so discovery
I love discovery v5 I need to make
stickers for this it's a big change from
each one you know an if one you used to
have just the URL being broadcasted to
all your peers and you would have those
kimia buckets where it would be
organized in Discovery v5 you now have
the notion of if am not records which
are still signs or still making sense
it's a little bit of security can do and
handshake so you can kind of make sure
not being spammed as much but what comes
with in ours is now the ability for you
to have in those key value pairs you can
arbitrary stowing them a notion of
topics so you can actually when you come
in you know podcast you stuff to others
tell that you're in if - node or you're
an East 1 node or you'll poke it out
node right so this allows all the block
chains to also participate in a global
DHT is nucleate immunity HT which is
also simpler than it used to be so it's
going to allow better security right so
like six months ago French came to New
York and one thing that I mentioned is
that dc5 will only work if you have over
thousand nodes running it otherwise the
security is going to be compromised by
eclipse attacks very quickly so this is
a huge effort going to allow if one in
if to to also share some of the work and
to share some of the infrastructure so
really excited about this is a network
geek so next steps for us so we're
recipient on EF grant we think we think
if M foundation for helping us in
sponsoring this effort we're going to
continue working on some of the tests
for SPECT conformance in terms of
networking so we're going to be able to
replay traffic send can request make
sure that we're getting the response we
expect across all the clients it's kind
of at the unit testing level if you want
and we've been talking along long last
about the lipids because if sub testing
so we'll continue eating that we want to
kind of have a hand in this make sure
that all the configuration parameters
are kind of optimized for the workload
of if to and we're pretty close to being
able to do a lot a client test met with
all the production ready clients does
the idea there is that every time
there's a new docker image for Trinity
or lighthouse or Artemis or anything we
can pick that up insert it into the test
net try it out see if it's behaving
correctly it's still able to find out
the blog sources good stuff
so not to put words into the if to
clients implementers mouth but their
heads down finishing up all the work all
the details of each cases all the things
that are kind of sticking out really
stabilizing the implementations and we
want to help them get the safety that
they need to declare facial release and
that they call your questions
you're asking how how things went and
how who started first is it a spec
effort or is it uh yeah so it's it's
interesting so there's been first like
some min Devine talking about if two
four years right so there's a lot of
like different initiatives and efforts
by vitalik eventually the research team
around if to kind of solidified with
Danny ryan kind of helping also get to a
point where he could name releases of
the spec right the spec was written by
Justin Drake to be more in a table
format so it could actually become part
of the testing I think Pato Liam that
really helped as well to really create
testing and tooling around it and in
parallel all the I mean the
implementation I'm most familiar with is
Artemis for example right and what
happened is I had a lot of churn because
lots of moving pieces came back and
forth right so SSD for example at three
or four incompatible designs to each
other until the point that they agreed
on all of this so every little piece of
the spec kept changing as part of
decisions down the road yeah networking
is not done anything so they're a couple
fees networking one is the wire protocol
we're getting to a point where we have a
good idea gossiping is actually pretty
easy because you just use gossip sub
just send all the blocks all the
stations that you have Susana is not too
much of a problem in terms of sinking
the the team right now are talking about
different sinking strategies if you
familiar with if one and all the ways
you can sink there are so many ways you
can do this right it's still something
that is an active research and our time
is up
so yeah there's more and there's no RPC
interface for if two yet as Greg
mentioned yesterday on this very stage
when we did the first round of research
we found that the median time was pretty
low and pretty pretty good but we had uh
pliers which were really really bad so
it looks like the routing strategy is
not always optimal I think that's the
only hunch I really have at this point
okay so how we scale to thousands of
nodes in our testing scenarios
yes so right now what we do is we do
small-scale testing about 30 to 50 nodes
our firstly p2p testing was 100 in
required quite a bit of compute right
now we are in touch with that research
labs came to give us pretty much
infinite resources across the u.s.
continent so we'll see how to go about
that we run docker containers so we're
pretty agnostic about the underlying
hardware we you know we run on
virtualized cloud software like GCP
Amazon Azure
so the idea is we just add more and more
machines about 100 nodes per machines to
scale to that we try not to do the other
islands of nodes from experience from
practical experience we study notes you
get already a lot of insights because we
can we can really tear apart all the
network connections like we can simulate
a lot of latency we can actually have
nodes being submitted between New York
and Tokyo for example to really show
what happens if the transaction takes a
long time to even come back to you so
second part of your question I'm gonna
get killed by this guy yes
you