[Music]
hi
hello so yes I have a long title uh so I
will be speaking about data Varity
sampling and uh it's a long title and I
didn't want to make it longer but
actually I have to make a small it's not
even blocks it's data variability space
that we are speaking of so let's see who
we are first uh I'm chabba I'm from the
resarch team from codex uh we are
working on uh decentralized storage
Network providing high durability
guarantees and our research team is also
working with thef on datability
sampling and also uh the client themes
okay so I think you are all familiar
with this chart you have seen it like 10
times during this Devcon I suppose uh we
had the IP 4844
and the transaction fees on L2 went down
right and on the right side we so see
those little little ones uh that's very
good that's very nice of uh
4844 but I hear that we have a problem
uh and uh yeah to see that problem I
have to zoom in sorry so if we zoom in
we see this problem uh so there is a
slight increase in R2 transaction fees
and actually there is a problem uh A2
transaction fees these are uh driven by
uh glob Space by data ver space and
that's running out it is problem and if
that happens prices can rise fast there
are other datability Solutions uh in
codex we are also building one uh but
here I am to speak of the ethereum
datail solution so we should scale that
ethereum data ability
solution so uh yeah let's look a little
bit behind these
numbers what you see here here uh maybe
you have already seen this one as well
this is the daily average size of blocks
uh on the network and you see that it
was rising and then uh after the merge
it started to rise and then uh with
4844 it uh
reduced so this looks fine
right now you see this less often uh
this is the problem so this is what our
aerators are seeing this is the blobs uh
so this is the blocks and the blocks
cumulative plot uh together so this is
the data amount that they are seeing and
now we can start worrying right uh there
is more data to handle but we are
scientists we can look at this different
way so let's see it on a log scale and
then it's not that bad actually like
it's like just like Mo slow it's
actually a different exponent uh but we
are better than those CPU guys so uh why
would worry
okay we just have to do that little
technology Evolution which keeps this
running and
then we have a solution right we can do
that line we just need more b space and
uh and that's a plan and here's a plan
we need p d and then four D and we just
have to be a bit lousy about the scale
so it's an unknown scale here unkown
scale there and then uh we saw the
problem what we want is more space and
what we want is less no traffic for our
operators uh yeah we just have to work
out these little networking
details so uh let's see those little
networking details that we have to work
out uh but first let me uh have a quick
the tool
networking uh but this is I was speaking
of average book size till now uh but
average is is a was a daily average and
in the day many things things happen
right every 12 seconds something is
happening so what you see here is the
tail distribution of block size uh
during a long period you don't even see
the small blocks you just see the large
ones uh and this was before
4844 uh and what you see that there were
blocks Beyond 2
megab and uh our notes have to handle
those what if two blocks of such come
one after another and you have to handle
that so you our technology have to
handle these things as well and we were
measuring uh whether you can handle that
and the good news is that uh you can
handle that so the network was was
running and live and what you see here
is uh for different block sizes these
are the bins Smalls up to 32 kiloby
blocks and then between one megabyte and
2 megabyte how much time nodes need to
get the blocks and the bigger the block
the more time is needed this is a
distribution so the notes which are
receiving it only here and there are
many notes which are receiving it before
everyone almost everyone is receiving it
before 4 seconds so things are fine uh
but clearly doesn't scale very well as
you seen if we want to increase the
block size then it would go to the
right so we arrived to our Point how to
scale layer one from the networking side
and we can do different things uh first
of all we can change the slot time the
slot timing window and the internal
timers uh we can change resource
requirements uh it's a good discussion
uh it's a bit controversial we have
small gains we have lot to lose so let
me not go into that one then we can
change uh the gossip sub the protocol uh
the networking protocol we can do
improvements on Gossip sub for large
messages and this is in making uh it's
important but this is not my talk today
uh and it's relatively small gains
compared to what we need still important
then we can do Shing and dat data with
sampling and this is my talk today uh
this is large gains I
hope and then there the four thing we
can do distributed block building uh
this is kind of intertwined with many
things uh but prises large gains but
basically that would mean resyncing the
whole data flow of The Blob transaction
from submission of the the transaction
Until the
End okay so let's focus on point three
the sharding and data sampling over the
network so uh data sampling uh let's
just key Concepts so you need to prepare
the data we were speaking of blobs uh
there are blobs there's a package of
blobs uh and then we need an encoding we
need an encoding because we want to
sample from it and for the sampling to
be effective we need AER coding for
theer coding we need segmentation so we
need to segment it chop it to Pieces we
need to extend it with AER coding then
we have to commit to it so that it
cannot be uh not send for PES basically
and once we' done this we are ready with
the data preparation okay and here we
have different structures uh we have
what we call the onedimensional P
structure where you have blobs which are
extended and through and committed to or
we have the two dimensional structure
where you have blobs which are extended
in two Dimension uh if you look
carefully you see that actually this is
also two- dimensional the difference is
that it's not extended by an addal code
in this dimension and that has
consequences okay and then you just have
to sample it sounds uh easy right uh the
question is where you are sampling from
and how you do this sample sampling so I
wasn't saying anything about that uh so
let's just see uh
that okay so we were typically speaking
of D Das as uh but it's actually a DA
plus an S so there is the da part which
is a property of of the of the network
it's a global property the data is
available the data is not available and
to have the data available in the system
I have to dispers it in the system so I
have to send it out different pieces to
different nodes which are custody it and
only then I can start
sampling so this is the da part it's is
uing data ability it is doing the Shing
not s only receiving pieces and then
there is sampling part once the data is
in custody at different
nodes uh you can start test sampling
asking for pieces and getting
pieces little uh graphical
representation so uh you have a builder
it does here in the example I was doing
it with the 2D encoding so it does the
2D encoding and then you have those
Beacon nodes in the system and what you
are doing you are sending out these
things uh pieces of this actually rows
and columns two different nodes someone
is getting only two rows and two columns
someone is getting a bit more someone is
getting the whole thing because he's
running hands of validators and he's
securing so much money but it's better
for him to get the whole thing plus
doing that he can support the
system that's one side that's just the
data viability now you have to sample
and that's an individual
node and actually every individual node
has to sample that's his own view so he
is picking a few
pieces and then asking for those from
custody so you're getting this piece
from here that from there that from
there and if it manages to sample then
things are
good
okay so we have this kind of two sides
of things and you can see that on the
network these are a little bit different
and we need them to
behave differently so we need them to be
secure robust fast and cheap and we need
sampling to be secure robust fast and
cheap um so we need the same properties
for the two things it's just uh
different
protocols
okay
so I was saying that there is peas and
there is FAS and there is difference in
the data structure actually there are a
number of differences uh if you scan
that QR code we have a right up on on
full which is starting from uh from pust
and uh some of these techniques apply to
pust some of them to fust so what you
see here is peas and in pillas you have
onedimensional
extension and that also means that when
you are sampling you cannot just get a
small piece because it would not tell
you probabilistically enough you have to
get a full
column so your sample is a column which
is more data than a cell
that you can sample if you have this two
dimensional
extension uh but to make this work you
need changes in the networking changes
in the other coding changes in many
things
okay
so let me just give you a comparision of
peas and
foras actually let me go back and and
just go through this list so we have
techniques here uh the first category is
about sampling is about how I'm
selecting what to sample how I'm
selecting those small pieces that I will
be looking for then the second category
is about uh AER coding and the using of
the aure coding because what I can do is
if the data is AER coded I can use that
as part of the transmission so when I'm
sending the data I can already use the
aure coding to recover data and then
send it on to others and that's a very
important property we are always using
aure coding when we are doing
transmission actually take your mobile
phone take whatever uh here we can use
aure coding uh very efficiently to send
data and when we are sending a row we
can take a piece and we can send it on a
column and then we we have when we have
received half of a row we can extend it
uh to uh to whole row for
example uh and then there are techniques
which are protocol changes and I'm not
going through all of them just showing
you the comparision so p and
FAS uh so one is cheap robust fast and
secure and the other is cheaper more
robust faster and more secure and I was
inclined to stop here uh but let's see a
little bit more in detail
okay so Fus is cheaper uh it's cheaper
because it's as I shown it's sampling at
the cell level and the cell is 512 bytes
actually it's
590 bytes because it's the data plus the
proof uh which is still much smaller
than than a whole column so whenever we
are uh so the
sampling uh becomes
cheaper it's more robust because it can
do arure coding which is allowing uh
what I was calling local repair but uh
what local means uh is important to
understand local means in the sense of
the code in the sense of let me go back
here in the sense of this Square so I
only have a few data I only have half of
a column and I can repair something I
can generate these or I have these and I
can generate these well I have a half of
the pieces here and I can generate this
one I cannot do that up there
and that's a huge difference that's a
huge difference because that means that
every single node can do repair not just
big nodes who are having old data uh but
every single node actually in that
construct we don't need big nodes uh
which by the way we have in etherum so
it's not like we don't have them uh but
we are not relying on them doing the
repair in the in the data structure and
it's faster why it's faster
uh because we are working on new
protocols and uh both for the pops up
part and both for the sampling
part
so and yes uh is it more secure uh so
the right size is still in
resarch uh should it be more secure
actually when this is secure enough uh
that we are changing to this actually it
doesn't uh it shouldn't be less secure
uh that's still in research this is
still in specification and changing so
uh let me not do more comparisions
because this will change and actually
this is still in
changing
okay sorry uh last
slide as I said it is not just peas and
FAS uh but we can go beyond and uh to go
beyond we have to resync the data flow
so how is this data
flow what is happening what I was
speaking about is what's happening in a
consensus layer so when a block is
generated how that's distributed but
actually these blobs are coming as
transactions from the world okay so
let's say one blob is coming
in and that goes to the execution layer
and the execution layer has the mol and
the mol is redistributing this sharing
this blob between nodes actually when
it's blobs it's not shared like when
it's normal for small transactions when
it's normal small transactions these
transactions are pushed to others when
it blobs is just information that I have
this blob and then the others are
pulling it overall
effect these blobs are spreading in the
network let's see there is another blob
that's spreading also in the network and
then there is one node which is
selected as block
proposal takes the
blobs packages them does the encoding
and then the encoded version is getting
spread in the consensus layer everyone
is getting his place I think you start
to see the problem right it's not the
problem it's an inefficiency we have an
inefficiency in the system in which we
are Distributing the data here and then
we are actually redistributing the data
there and is the same data so we can
optimize it a lot which is good right we
need these optimizations for Mo roow to
happen so uh we found another place
where we can save a lot and have that
curve go up so what you can do in first
instance is that we do the distribution
up there but whenever the node is
realizing that he has a piece he would
need this and this but he has the blobs
he has the data he can just pop it up
from the execution
client because the data is also
there now of course this plot is
optimistic because I was putting this
white blob almost everywhere and I was
putting this red blob almost everywhere
in the network in reality you don't have
every blob everywhere uh in the
execution layer because if we we would
we could do that like now you actually
have but when we are
scaling we will not be able to do that
because if we can do that then we don't
have a problem every note can handle all
the data which is there we want to scale
beyond that so in reality you would have
some blobs here some blobs there some
blobs there but then when you do that
distribution data can pop
up and uh that's one of the
optimizations of course that's a simple
one we can do much more complex ones
where we have interaction between the
two and we are avoiding some of these
duplications uh but for that we have to
resync uh the networking stack here
which is that P2P here which is using Li
P2P they are doing different kinds of
gossiping distribution uh this is using
gossip sub this is using other protocols
so this is all uh this can all be
Ling uh but that's just
starting okay thank
you okay we are now here to take some
Q&A um so starting from the top there
are blob fees that play blah blah blah
blah blah blah there are blob fees that
pay for blob storage but is this
verified at all what happens if I get
the fee but don't actually Store The
Blob
data um okay blob fees that pay for BL
storage I don't know
yeah I'm not sure what the question is
about no worry question answer question
asker if you have the question still
feel to clarify clarify
um but I'll mark this as answered what
are standing problems that hinder
current
adoption okay so
um it's just in implementation so the p
is in implementation uh it's still
compatibility Problems P will be here uh
very soon it will not be in pcta but it
will be in the next for uh for FAS we
still have to figure out uh many things
so uh there was that question mark on
the on the on the security uh and we
have to work on that to make sure that
there are no issues with
that cool why do we need 2D eraser
coding can't we put all of the points on
one polinomial and extend it so putting
them on one point and extending it would
be uh one huge uher code uh one
dimensional and that would be very bad
for repair in the sense that then you
can only repair if you have all the data
and that just uh means that
uh you have difficulties repairing the
2D has this advantage of having this Cod
local repair property so that the code
distance is small and you can uh you can
repair pieces so you can repair a single
column you can repair a single row and
that gives notes the possibility to rep
start repairing and contributing even
before getting the food
data if full dos is way better than pure
dos why not choose to implement it in
the first place uh uh so as I said uh
buas itself is changing we have we were
thinking of techniques for FAS then we
realized that they are actually applying
to pilas and then they are part of pilas
uh we're still changing that uh FAS is
still much more resurge uh it's a simple
fact of life that uh there's too much
pieces still moving in F us uh to
implement
that Co and to cap us off it seems FS
approach are much much better than pios
should we Skip pios and imp the FAS is
there any other
tradeoff I so FAS is using many things
that pus uh does so skipping is is not a
a direct Skip it's not fully different
uh it's it's giving the 2D C so it has
advantages but it has also complexities
uh whether we keep or not is uh might
save something on the implementation
uh when we introducing it it might also
skip something but we are losing also
the graduality so overall I'm not sure
it's good to skip uh but we can uh we
can think of it once food us is fully
evaluated what are open research
questions so uh the open research
questions is uh so the last slide was
was was fully open uh research question
or everything around that uh the on on
FAS uh there are several questions
around the sampling how we can do the
sampling safely in the sense that c
nodes cannot attack the system for
example so uh there are still things to
clarify there uh especially attacks
Network partitioning and other things
how they affect
security there's some fighting and up
voting going on this is fun if 2D is
better would there be any benefits of 3D
absolutely uh multi-dimensional encoding
is fun uh but no actually not so we just
need the uh the local reparability
property and we have enough of that in
2D 3D would not contribute to that uh
too
much uh can fust be implemented
progressively I think so but we didn't
Define plugless so uh it's something to
think of but uh but we didn't Define it
yet cool then I think this question was
already answered I think you already had
that right yeah unless someone's
aggressively or I just forgot to mark
his answered um cool so we have a minute
and 34 seconds left I was actually she's
spending time yeah no but so I guess
there's one piece to this question you
can give him feedback on the QR code so
we'll give you some time to tell him how
great he did collect a card um but other
than that I think our next
session um well I guess