foreign
[Music]
Workshop about running the beaten chain
Explorer on your own
um we will go through
how we came to the idea to make the
beginning chain Explorer and then we
will explain how we how you can actually
run the beacon chain Explorer we this is
Stefan
my colleague and me Patrick
we both work for the company pit plan
so yeah first we are
Stefan will introduce you
to our company and how we actually came
to the idea to make and explore to the
for the next stage of the ethereum
project
then we go briefly into the through the
architecture of the project and finally
we will show how you can run the
Explorer on your own
but before we start with anything
everyone who everyone who wants to
participate like interactively we need
you sorry I was on the wrong laptop
yeah
we will go first to how we came to the
beginning expert and what what the
architecture of the project look looks
like and finally we will show you how
you run the Explorer but before we go to
anything we need you anyone who wants to
participate we need you to to download a
few things
so
are there any anyone who wants to
participate on this laptop
um do we have install Docker in Docker
compose
so can you basically try and see if you
can access the repository if we made it
public
or not
yeah then
I'll quickly we'll jump back to that
and I'll quickly introduce kind of who
we are a bit more in detail
so um the company bitfly was founded
2017 and we've done a range of products
um the we've been basically active since
the beginning of ethereum in this space
we actually created one of the first
block explorers but kind of let it slack
a bit so that was etherchain.org it's
one of the older ones and then later on
we kind of regretted not putting more
effort into that Explorer and then we
kind of decided now that there's uh
gonna be ethereum 2.0 or
um basically the merge we decided to
create another Explorer specifically for
phase zero and that Explorer kind of
ended up being the beacon chain Explorer
and it started out where everyone could
basically see all the information about
the beacon chain uh maybe you shall of
hands who has used the Explorer before
maybe just to kind of get a feel
so about half of the people have used
the Explorer before
um
so maybe if you want to show off hand
who's running a validator
okay uh quite quite a few of the people
that are using it
um so basically we we wanted to create a
nice place for you to understand what's
actually going on with the validator uh
to see the different states of the
validator to kind of make sure you don't
get slashed and we also operated uh one
of the bigger mining pools called
ethermine.org that was now retired since
the merge and we kind of shifted away
from mining a bit and focusing we're
focusing a bit more on staking so kind
of to chill a bit our other products we
have staking.ethermine.org now where you
can stake with less than 32 each and we
have eat pool if you're too lazy to run
your own validator you can just upload
your validator key and we'll kind of
take care of that for you
um
uh and then a brief history I kind of
already touched on that so um it started
out in 2019 end of 2019 with the first
test networks so even before
um the Genesis of the beacon chain here
you can see a small or kind of nice
picture of the Genesis event
um we kind of created like a slot view
so each one of those rows if you don't
know is an EPO and in one Epoch there
are 32 slots and here you can kind of
see the green part is when it was
proposed the red one is when it's missed
and the yellow is if it's orphaned and
that's kind of a nice overview of the
first few epochs when the Genesis
happened and we also have like a
checklist below how much finalization we
have uh no our participation we have and
if the EPO has Justified and finalized
and a nice rocket or like the countdown
when Genesis started which is also
pretty pretty nice so that's kind of uh
how how this whole project began and it
kind of exploded a bit more and more and
with the merge we uh tried to add more
information not just
um the phase zero information and it
kind of grew and grew and this
presentation will also be kind of about
how we kind of handled the scale and
also give a kind of a workshop where you
can help us with with kind of finding
ways to scale better
um yeah so let's go a bit to the
architecture so in the beginning this is
kind of a very simple view what uh the
beacon chain Explorer looked like I also
presented this a similar slide I think
it was etcc a year ago where we had just
the prism node we had infuro node and
the funeral node was mainly to get the
deposit data from East one and then we
had an exporter that just basically
wrote everything into postgres database
and that worked really well in the
beginning it was really easy to work on
and then we just had a golang front end
with some templates that kind of served
everything to the end user and even when
I presented it back then we kind of can
see that prism already needed like a
576 uh 67 gigabytes of data Erica node
already 1.4 terabytes and cause postgres
already two terabytes of data and then I
looked at these numbers again how it's
running today and kind of the reason why
we had to scale a little bit more so the
numbers now are that we switched to
Lighthouse and we have a 32 slot sync so
there's different kind of sync versions
that you can choose and depending on
what amount of stats you choose the more
information is stored but the quicker
you can retrieve information and we
basically want to store everything and
that's already a five terabyte disk that
you need for that Lighthouse then Aragon
is 2.1 and our postgres database is huge
so uh kind of the tables and the indexes
that we have in the postgres database
they just don't scale that well so it's
already 10 terabytes big and we don't
only run the mainnet we also run the
test net and the test net is just that
amount and even a bit more for different
test Nets so it's a huge amount of data
and it's getting a bit expensive running
that in the cloud so we're going to talk
about how we kind of started migrating
away from that
um and basically the the scaling
challenges uh during the merge so this
is kind of a analytics view that we had
so we had
um concurrent sessions of 2267 uh people
that were looking during the merge
because we had like this nice slot view
that you saw before where people could
see okay during the transition
um what was happening uh it was it's a
nice visual way of kind of tracking it
and we had a lot of people
um we kind of were kind of prepared so
you have this Super Mario mushroom that
should power you up but that mushroom
was kind of overpowered by the people
looking at it because uh the way the
architecture was designed it just didn't
scale well we have front-end instances
that kind of have to query the same
stuff like if we have five front-end
instances they do five times the same
queries because we don't just have like
one layer that updates our cache we just
have every front and instance that has
its own cache so these are things we
kind of try to try to improve and then
again we have very expensive indexing as
well in bigtable and big I know a big
table that's going to get to later in
past Chris and pascals also has to get a
lot of the table into memory to be able
to query stuff
um which is a lot of the times not very
efficient especially with huge amount of
data
and
um
so what we try to do is we have we
started out with one big binary where
everything runs and we try to strip away
things from that big binary and kind of
isolate that in microservices that not
every binary does everything and if we
duplicate one binder we don't do uh
unnecessary work
um and kind of the migration is kind of
a bit challenging for us because then we
have to kind of make sure that we manage
a lot of the technical depth that we uh
kind of have and are maybe adding
through changing Technologies
could someone maybe say if they can
access the repository yeah yeah it
didn't there's a Docker file in the
docker compost file in it and if you
just run Docker com talk compose tool
it will pull all the images
and when we later get to interactive
part
everything will be done
we have all the things ready
okay great I hope the internet is good
let's pray a bit because when we tried
it or tested it before uh it took a
while so um here you can see I kind of
touched on that
um the scaling and large table so in
postgres in the beginning we didn't have
any partitioning at all so later on when
you can just maybe hold out I can hold
it up and you can maybe
okay
um again lots of assumptions that we
made without knowing too much of the
spec or or your uh specific use case
let's go so Cloud first I think still
makes sense especially for like dos or
whatever you guys might need that for
um in terms of front end I don't know if
this is a bottleneck but we mentioned uh
thought about clustering or some Auto
scale for this or even if we're going to
the web3 ethos of let's put this on ipfs
or like distribute it somehow
um and merely make it un
undeniable Service uh scaling that way
or it could be just some some Auto
scaling surf Some Cloud run or gcp or
whatever I use for that so I don't think
this would be the bottleneck anyways the
front end itself I think that and if
depending on what you have here this
could be heavily
um statically generated so it might be
something that you wouldn't be a problem
then uh second layer we're talking about
cash so I imagine most people are our
team imagines that most people are
looking at past data so you probably
don't have to have uh direct access to
the database you don't have to do like
real queries against it you like cash
heavily previous data that was already
from previous blocks that will never
change so things that could be like
pretty much Frozen there because it's
not it's really static and and and and
stale deal it's called like that
um so this depending I guess there's
probably lots of ways of doing this
catch layer this could be like a DB
level this could be some like redis cash
flow or something like that but yeah
really really something before the data
business itself then on database
um our friend I forgot his name but our
friend there give idea of maybe we can
start sharding this maybe some sharding
based on transaction hash or something
like that which
um same thing since this is mostly data
that will live forever and not touch it
again we probably have a part that's
only read only so it's there um very
high output for reads uh not that much
in terms of Rights and a part which will
be the right part which this part here
would have more direct access to the
actual nodes and then the nodes
um probably make sense have multiple
archive nodes if if the RPC calls my be
a problem I don't know if it makes sense
to have lots of different machines
running those instead of inferior
inferior for us in our company is always
a problem and turn off if there's anyone
from these companies talk to me please
but um yeah maybe just run your own
archive notes for these not sure also if
the actual RPC calls are a a bottleneck
for you guys no it's good to have a
backup okay so at least yeah maybe to
like archive knows you guys are going to
run for that
um and then this part here would just be
like the heavy right part which
everything that you checked as a new you
would just update database based on that
thanks so much okay thanks for
participating
[Applause]
I'm gonna hang that up okay
so
just a solution we came up with
so we we switched kind of the solution
we have right now yeah this is the
solution we have right now
uh with which we part of the of the of
the of the data set we now store and
pick table instead of important postgres
so we thought like it's not really
necessary to have a relational database
for this kind of data set
and bigtable scales really well so I
think that we we were able to
to indexed uh all of the current uh each
one transactions
uh in five hours so it's like really it
scales really well depictable
so we now store
the blocks
the balances of the validators every
transaction
all of the attestations in bigtable
um
and this resulted in a really much more
stable
um in index time so now you can see it
takes us
like really Stables 30 seconds to export
all the data of one epoch
yeah this is where we come to the
interactive part of this of this
Workshop
um for everyone who wants to participate
do you think
so he's just resetting everything he's
done to kind of start fresh with you
guys
so the first thing we we want to do is
to initialize the databases
so
we turn on postgres where it is in Peak
table and initialize the the postgres
tables and picked the big table what is
called tables
uh database
so
the next thing we want to do is to start
it one indexer we called it that because
we're not really creative
but it indexes like I think it it first
um
gets the blocks
from the execution glands
um and encoded in protopath and still
using into big table encoded in protobuf
and then
we index as after that we index the the
blocks with all the the fields and
values
and then transaction as well so
one
one big thing is like we're actually not
using bigtable from from Google chart we
are using
emulator from the company P key
what's up
it just emulates the the peak table API
and stores it in sqlite database
and I think like for
for just storing and then analyzing the
data it works well it's not like it will
not scale for uh
for a thousand requests per second but
for analyzing the data it's a like a
really good solution solution in our
opinion
so the the thing here is
we are not able to
to reach the nodes we we think before
the the workshop in the cloud somehow we
don't get access we have this wi-fi fan
so we are now running the nodes
on our laptops and that is where you see
some errors because the nodes are not
synced and it it's not like doesn't work
well but
foreign
[Music]
index around me the next thing we want
to run is the
exporter
the exporter is like truster and the
export we had before the merge
but now it will index the
the blocks from the
from the from the big table that we just
exported in the in the 81 induction
again we have some errors because the
notes are not synced
the next thing is like the the
statistics module module and this is
also a really important uh and integral
part of the whole project because
the indexer will aggregate the data
one day
and store the aggregated data of the one
of of one Beacon chain there in the
database and so it all the queries
become much faster
the last thing we start is the front end
and we'll get started with the
with the cache up data and then the
front-end itself which is a golang web
server which just queries all the
databases so all the things are running
now
and we can browse the pink chain
Explorer
on localhost 8080 the thing is
since the nodes are not synced
um it will take some time until any data
will show up
um
this one
there's some extra extra scripts we
we wrote in into the into the into the
another main script um
the thing is like we we understand like
this repository not as we want to to to
make it the big change for available for
everyone who wants to to explore the big
engine not
so you don't have to trust us
by going to Beacon chain uh we can begin
chain we can change beacon.in
you can run it yourself
it will take some time until it's like
synced and we will
we will try to make it more easy
but the thing is we really
um what does it saying
We Stand by the decision to make the
secure open source so it's not we will
we will try to to make it available for
everyone it's not like we made the
decision to go into bigtable from from
Google and to run it in gcp just to
to make the scaling issue like gone
but that doesn't mean it's it's not
available for everyone
thanks to such amazing open source
projects like from bitly the gcp
emulator
so the last script is
now you have like a postgres database
where you can think with the data
we made just some
hmm
the script is not uploaded
no it is
yeah okay
let's restart the services
I don't know if I want to do that
has anyone followed a lot is anyone
following along
is it working well okay it's working
yeah
yeah okay okay okay yeah
so so the thing is like with this uh
repository we show really how you can
set up the Explorer and we we know it's
not like there's a lot of technical
technical depth that is like
we need to solve it but it it's working
for everyone not just for those who have
really gcp accounts and everything
um
I think we can go to the next slide
yeah
okay so for anyone using
um our mobile app or the website we've
got a discount code Bogota 50 if you
want to use it you can get our Premium
app
Services uh if you have a lot of
validators it's nice uh among other
things or if you want to use our API
Services you can use the code over the
website not directly over the mobile
um and yeah if you're running a
validator the mobile and you don't know
about the mobile app it's pretty nice it
lets you look at your validators on the
go it lets you get notifications when
you propose a Blog
um and kind of see if you've gotten uh
those 30 East block Rewards or something
like that
um yeah
okay then
um
yeah if you want to contribute
um here's kind of the overview
um of kind of the resources so
um we have a good coin Grant that's
linked below uh you can tweet at Peak
and chain if you see any issues or
something create issues uh on the GitHub
repository that's also linked there
um and
oh did anyone still want to take a
picture
or no thanks yeah that's basically it
from us thanks so much for participating
and listening
if you have any questions
um yours right now that you personally
it's called so it's one of the test Nets
it's a bit smaller so it should be able
to think in a reasonable amount of
amount of time
um the Wi-Fi isn't the best here so here
it could take a while
um but it's a lot smaller than Plata or
which is girly or mainnet which are
pretty huge right now
okay any other questions
um
not sure if I if I should answer that
but
um it's it's quite a lot it's quite a
lot so it's
um what is it
five figures
we looked especially
especially but we looked also at other
DPS
but in the end we took something that
also fits in our team
like we don't have so much resources I
was like so much
take so much man hours to get into a new
technology
and
for us this looked at the best fit
and right now we are really happy with
it so
yeah we basically went from an export
time of 30 plus seconds for attestation
assignments to like five seconds which
is a lot better and there was less
contention between different queries uh
on bigtable now that we moved but that
can change if we have everything on
bigtable we're just starting to migrate
so I hope it stays good
yeah and Layer Two is coming yes okay
anything else
thank you very much for listening and
for participating again yes thank you so
much