[Music]
hello again so this is going to be a
longer talk the intended audience are
developers who want to build scalable
and responsive distributed applications
on top of swarm using DNS so first I
would like to talk a little bit about
the distributed application paradigm
itself because in many ways it's a very
radical departure from the client-server
model that is prevalent on the Internet
today and with which most web developers
are intimately familiar then I'm going
to talk about the various challenges of
scaling and what the bottlenecks are
then we are going to discuss how
reliability issues are being addressed
in a dynamic decentralized and
essentially low trust environment of a
distributed application we are going to
discuss availability of content which is
a very important issue and finally I'm
going to talk about how a distributed
application can be operated and
developed and improved over its
lifecycle
so first the distributed application
paradigm distributed applications are
much more heavy on the leaf nodes so
endpoint nodes perform a lot more work
than clients in a client-server
applications very large part of the
business logic are actually done on the
client side so on the slide you see
several possible
nodes which have their own requirements
but which can be the nodes in a which
can be the nodes of a distributed
application so it's not necessarily
web-based there can be in native mobile
applications Internet of Things devices
and also we need to keep in mind that
key and identity management is sometimes
outsourced to this hardware keys and we
need to take that into account the
backend is typically generic and I will
talk a little bit later about the deeper
reasons behind that but the kind of
backends on which we rely are the ETH
and less protocols for consensus so it's
either a full node or a light node on
the etherium network so each node needs
to be an ethereal node we use swarm for
permanently storing large amounts of
data and accessing data and we use
either PSS or Whisperer for node 2 node
communication so all of these backends
are entirely general-purpose they're not
application specific and even though I
told you that there are no clients and
servers in a distributed application you
should still keep in mind that the
contribution of resources is by no means
equal so peer-to-peer applications are
not necessarily not necessarily balanced
in terms of the use and contribution of
resources so even if you think about
BitTorrent there are Cedars and leechers
and in the general terminology
we distinguish between consumers and
suppliers of resources and of course
each participant can choose and even
change over time to what extent they
supply and consume resources but
typically consumers are are
characterized by high churn so they
leave the network and join the network
frequently you cannot rely on them
staying on the network for a prolonged
period of time typically they spend
accounting units whether they are ethers
or special-purpose tokens and they have
resource limits which in some cases can
be quite severe suppliers are typically
there in order to earn accounting units
so they are providing a service and
expect to be paid for it they have a
launcher so they can still leave the
network and join the network but they
don't do that frequently they have high
availability and they have adequate
resources for providing the the services
that need to be provided for this
distributed application so in the next
part I'm going to talk about the
challenges of scaling distributed
applications and identify bottlenecks
and proposed ways of dealing with them
so the most important bottleneck is the
blockchain of course we have heard
wonderful talks about how scaling
problems on the blockchain are going to
be solved over time and in a ideal world
in which we had a essentially infinitely
scalable blockchain
everything I'm talking about would be
unnecessary because we could host the
entire distributed application on the
blockchain keep all the data on the
blockchain and submit every transaction
that the application ever does directly
to the blockchain
but there are two reasons why I feel
that the techniques that I'm going to
introduce are still relevant one of them
is that the scaling of the blockchain is
a slow process and it's a much harder
much harder challenge than scaling the
particular dab bottlenecks that I'm
going to talk about it so people want to
create scalable applications even before
all the problems of blockchain scaling
are going to be solved and secondly
because the techniques that I'm going to
talk about are actually useful for
scaling the blockchain so for example
the initial motivation behind swarm was
to store his historical record of the
etherium blockchain so archive nodes
which store the entire blockchain
history as far as I know there are very
few of them maybe none and but we still
want to keep the entire history of
etherium available and swarm is the
perfect vehicle for that so all the
techniques that I'm going to talk about
can be used for scaling various aspects
of the blockchain so on the slide you
see in what ways the blockchain
constitutes a bottleneck so because all
information needs to be replicated over
every node that means that storing
information on the blockchain is very
expensive so you don't want to burden
the blockchain with too much information
also the blockchain can only be updated
at the block time speed which is on
average a dozen seconds or so which is
not exactly responsive and you have to
pay for submitting transactions to the
blockchain so if you do updates very
frequently or your updates are very
large then it again becomes overly
expensive the other bottleneck that we
are going to encounter are individual
nodes so basically you don't want
individual nodes to perform all the work
of a particular area of the distributed
application so you want to distribute
all the tasks of ADAP or as many tasks
of the app as you can over a large
number of nodes that are working in
parallel and also bottlenecks can be in
network links so for example you do not
necessarily want to broadcast every
interaction to the entire network
because then you're going to have a
network congestion and in case of
systems like whisperer you may need to
pay in terms of proof of work for doing
so so these are the bottlenecks that
we're going to be dealing with so the
storage bottleneck is best overcome by
storing everything that you can inform
and put the root hash of your data in
iana's so the root hash is just 32 bytes
and it can integrity protect as well as
identify data of arbitrary size and
complexity and structure meaning that no
matter how much data you're updating you
still on the blockchain you only need to
change 32 bytes which is obviously a lot
cheaper than changing megabytes
the transaction bottleneck can also be
overcome so for those of you who don't
know EMS has resolver contracts so each
time you ask Ian us what the particular
address corresponding to a name is
you're going to have a contract answer
you and those contracts can be more
complicated than a simple storage of 32
bytes in particular you can implement
Radin style updates on els meaning that
you can send those updates to all the
interested nodes but you don't have to
commit all of them immediately to the to
the blockchain and yet all the nodes
that are interested are going to know
that the content has changed and any of
them can commit it to the blockchain in
case it becomes it becomes critical that
a consensus is reached so this way we
take the burden of updates of the
blockchain and we can have in frequent
updates to the blockchain and yet have
immediate and very very frequent updates
to the root hash of the data that we are
using broadcast bottlenecks
they can also be overcome using PSS so
just as Louis had explained before me
PSS can be used to broadcast messages to
parts of the network instead of the
entire network so if you have this
pops-up model then PSS will make sure
that even though there is an overhead
you won't burden the swarm network with
every message broadcast it to every node
of course this only becomes a problem if
your
application scales really big so for
many applications broadcasting is
perfectly adequate and if you are more
interested in darkness you can still use
whisper but then you will have to keep
in mind that you have a network
bottleneck so in the next part I'm going
to discuss challenges of a trade-off
between responsiveness and consistency
it's a trade-off
because the faster things happen the
more difficult it becomes to agree in a
timely fashion so the consistency is
only guaranteed on the blockchain
but this consistency is a eventual
consistency which is rather slow so you
can see a back-of-the-envelope
calculation that tells you that you will
reach consensus in a time that can
hardly be called responsive and you
don't want to wait that much time for
every update to be visible to all the
users of your application
so essentially you need to broadcast the
updates to all the interested parties
and then you only need to commit to the
blockchain once it becomes important to
to guarantee the consistency over a
longer period of time so here I would
like to give you a few examples so one
one is a online discussion where there's
some kind of content be it a video or a
blog post or a picture that several
users are viewing and commenting and the
other is a massively multiplayer game in
which different players are interacting
with each other obviously submitting
everything to
blockchain is hopeless in both cases so
in both cases you can notice that it is
relatively easy to scope the circle of
interested parties of a particular
update so in case of a online discussion
it is perfectly enough to make updates
fast to those who are actually viewing
that discussion who are participating in
it and it is acceptable that others who
are not browsing that particular
discussion right now will only have the
update much later and they are only
interested in the final state of the
discussion they don't need to receive
every update at all times and similarly
in online games there are typically
locations and there are some players in
that location and you only need to send
quick updates to those players whose
characters are in that particular
location you don't need to send updates
to all the players and the update
structure is also more complicated than
simply a hash because if all you
broadcast is the new root hash of the
swarm content then you're running into a
responsiveness problem that it takes
time for swarm content to sync even if
we will improve and we will improve the
performance of the syncing protocol it
will never be as fast as a direct
message passing simply because a much
larger amount of data needs to be
transported and it also needs to be
stored along the way
so if you only send the root hash then
the response time is going to be much
longer than the message passing time
because you also need
wait for the corresponding sworn content
to become available for the interested
parties and instead what you can do is
you can also include in the updates the
actual information that was changed so
for example if a comment was added then
in addition to the new - you also send
around in messages and PSS messages the
actual the actual comment that was added
so that the nodes that are monitoring
the discussion will be able to
immediately update their local model and
yet they can verify whenever the swarm
of date happens and especially when the
blockchain update happens they can
verify that they haven't been lied to so
this does not present a security problem
it just speeds things up and also I
would encourage you instead of creating
very complicated update rules and then
you need various ways to actually verify
that the updates were legal or so they
were the kind that is allowed instead
what I would suggest is to do
aggregation on the on the front end side
so for example if we're looking at a
online discussion then every participant
would update the route hash of all the
comments that they have sent to
different parts of the of the service
and whenever a participant looks at a
particular discussion they should gather
the participant the addresses of the
participants of that discussion on ENS
and then monitor the updates of those
participants and independently of each
other and do the aggregation in
to a discussion themselves on the
client-side so that's a much simpler and
much more reliable way of dealing with
concurrent access in the next part I'm
going to briefly address issues of
reliability and availability so here I
would like to go back to the things that
I said about about distributed
applications whenever possible you
should not try to build your own
infrastructure try to rely on generic
infrastructure
because that way it becomes somebody
else's problem so in order for a
distributed system to work well it would
it needs to have a sufficient number of
nodes those nodes need to have
sufficient incentives they have they
need to have adequate resources each and
all these things they are difficult so
for example if you only use swarm as a
generic information storing framework
then making sure that swarm content will
be available is our problem as
infrastructure developers it's not your
problem as an application developer and
in general so there's a reliability kind
of ranking between these services so ETH
is the most reliable light client
protocol is less reliable bzz is even
less reliable and PSS is the least
reliable of these generic services
simply because of the popularity of the
of that particular service on the
etherium network and if you do something
yourself it will be way way way behind
all
these on the reliability scale so also
you should keep in mind that the only
thing that is certain is the blockchain
everything else is ephemeral so if
there's some important state update
eventually you have to make sure that it
reaches the blockchain and finally you
should be paying attention to incentives
so in case if you want to enforce
compliant behavior then the two tricks
that can make things more scalable is
instead of using pre-emptive measures so
writing complicated contract code that
can only drive the application mechanics
as the smart contract executes that
block time instead you can use reactive
security that everybody updates the way
they the way they can locally however if
there's some dispute then you can turn
to the blockchain and figure out who's
right and who's wrong and this way if
everything happens correctly you rarely
touch the blockchain so that helps you
scale so instead of using proactive
security measures you use reactive
security and the you only bother the
blockchain in case of disputes and the
final part of my talk I would like to
introduce some ways of maintaining and
improving and developing and adding
feature to a distributed application it
is very different from from the
client-server model because in a
client-server model you can do
everything on the server and you don't
have to
consider the fact that your application
is not entirely under your control so
when you're rolling out a new version
obviously you need to update ENS to
point to the root hash of the new
version but also you need somehow to
notify all the active users of your
application that things have changed
because otherwise they just keep
continuing using the old the old version
and it might even interfere with the
workings of the new version so in this
way it is very different from
client-server application also with apps
we have the same thing as what we have
with block chains we can have Forks it's
entirely feasible that somebody likes a
particular version over that and they
register it under their own iana's
domain and people keep using that so
whenever you introduce a change that the
community disagrees will or at least a
large part of the community disagrees
with you will have Forks I think I'm
running out of time so the last thing
that I wanted to mention is analytics
which is much more difficult than in a
centralized environment and it's ADAP in
itself that will be eventually developed
but unfortunately I ran out of time so
if you want to talk about analytics
please see me in Saturday at the
breakout session thank you
you
[Music]