[Music]
yeah hello everyone I'm very happy to
start us off on the update from the
swarm team so today we're gonna talk a
little bit about well what swarm is just
make sure we're all in the same page
what the status of the swarm project is
and also where it's going we've got some
new sub projects it's going in different
directions and the general scope of the
swarm has grown and so we're going to
talk about where we're going but I'm
just gonna start you off with what is
swarm to make sure we're all you know
understand each other you probably know
swarm does a network of computers share
data in a decentralized way to give us a
peer-to-peer decentralized storage with
all the benefits that come from from
being a peer-to-peer network such as
resilience against network outages DDoS
attacks and the rest but what you may
not know it's really the same
connections as etherium so same with
Nazareth because right now swarm client
is still separate but the vision is that
it's part of the same package of
protocols that would run over there p2p
so your swarm know your eath node could
talk the eath protocol or the swarm
protocol all of the same connections so
it's really tightly integrated with
aetherium so let me give you the basic
first use case the sort of baby example
of what swarm is for the decentralized
storage of app data so the blockchain
the iam blockchain is great to store
critical information about your DAP
maybe user name registrations or
something like that but your dad still
needs images HTML Javascript and all the
rest and right now you probably need to
host them on a server
and swarm is there to allow us to have
this in a completely decentralized way
so here's how that would work you'd go
to your swarm enabled browser and type
bzz which is a swarm protocol the name
of your DAP eath then the theorem named
service would turn that name dot eath
into a hash
so the ENS record has an address field
where you can store your wallet address
and that's what address would get paid
if somebody sends payment to that name
but the standard ENS resolver also has a
field called content and in that you can
store a hash that's referencing content
on swarm for example so you use this
hash of content to query your swarm
peers for the data and in return they
will supply the data to you and anyone
who supplies you with data that you want
you compensate them using a theorem
based micropayments and then the last
step is to just use that data to render
your DAP so the hash gives you integrity
protection you don't care where the data
comes from you know it's the right data
and gives us a server less completely
decentralized way to host Bond apps so
that's great so let me also tell you
what's happening under the hood on swarm
when you have a file or any content
swarm is agnostic to what the content is
it's gonna take anything you throw at it
and break it into little chunks but for
kilobyte each and the hashes of those
chunks are collected into a Merkle tree
everything is murk alized and the root
hash of that Merkle tree is the ID of
this file or this content in swarm and
the chunks they're all being sent to
different swarm nodes for storage what's
really happening is that the hash is the
ID of the chunk and every chunk gets
stored with whatever node has an address
which is closest to that chunk hash so
really spread out the data and your DAP
data is uniformly stored throughout the
swarm network
the nodes share services with each other
it could be data chunk retrieval and
serving data or other services but so
the accounting how much you have to pay
how much you get paid is done by this so
called swap so here's how that might
work
suppose node 2 is asking for a DAP and
node 1 happens to have the chunks in
question and gets to serve them so you
know like that it's my own animation I'm
proud of that and if there's more than
one chunk both nodes keep count so there
might be two four six after a while if
node one is not consuming any data node
two is doing all the consumption no two
will have to pay node one so it would
use our checkbook payment contract to
send a check to node one at that point
the channel we call it balanced you know
whether it's chunk four chunk or check
for chunk doesn't matter and when it's
balanced we can go back and continue
consuming services so what's happening
is in within this form a swap channel in
the middle where zero balance neither
node owes any node anything and as soon
as services get consumed the balance
moves off towards one side at some point
you're gonna hit the payment threshold
and at that point the node that's in
debt would have to issue another check
to the node to the debtor so that to
bring the balance back at zero so you
know what's always working to keep the
swap within that middle range and if you
slide too far off to the side well then
you get disconnected that's the
punishment the ultimate punishment in a
peer-to-peer network nobody wants to
talk to you anymore this in payment
infrastructure that swap is a part of is
very modular it can go in many different
directions it was originally designed
for data delivery content delivery and
storage long-term storage as described
in our orange paper but it's it's become
much more modular
and we're gonna extend it and we're
talking about this on Saturday in the
talk called swaps we're in swindle games
where we describe just how how this can
learn well how I can accommodate all
kinds of different services beyond just
storage
the status well some of you may have
tried swarm already if you have tried
swarm you've been playing around with
our release POC 0.2 and that really is a
proof-of-concept release it's not a
final release so the the
proof-of-concept showed us that the
basic idea works and we've had some fun
in a hosting content and trying it out
and playing with it but the performance
has been really slow we've had data
availability problems throughout the
network data that hasn't reached where
it's supposed to go but so as a
proof-of-concept it was successful and
the testament has taught us a lot and as
a result several of the core components
are being rewritten or have been
rewritten completely yes and
additionally the scope of the project
has grown as I've mentioned before
people are using swarm not just to host
content but for example the live peer
group are using it for broadcast
streaming video and audio really cool
they're also talking on Saturday another
project that's developed is about
communication direct node to node
communication and to say tell us more
about that I'd like to introduce Luis to
talk about PSS
[Applause]
hello everyone I'm Lois this is my first
Devcon
actually I'm a classical pianist turned
cultural magazine editor turned funding
bureaucrat turned band manager turned
taxi driver and I was privileged enough
to get a ride with some blockchain
people in Berlin where I worked in my
taxi and then told me all about that
centralized consensus war on privacy and
the looming dangers of opaque
algorithmically driven societal
conformity and this brought me back to
the world of computers that's I've been
in all my life in varying degrees now I
work for Jack part of the Jack team Jack
is a company first of all I very
clumsily forgot to include the logo of
Jack here but that's the logo they're a
sponsor we are a sponsor so Jack built
son swarm and we want to contribute to
swarm as much as possible of course and
therefore I was very excited when Viktor
Tron came to me and and challenged me to
do the PSS work and that's what I've
been working on the last seven months
now notice that among the things I said
I was I didn't mention public speaker so
that's why I have this not to panic I
will be looking at more at this than
it's you don't take it personally please
now swarm is a network of nodes and not
only that but swarm is network of nodes
that speak the same language this
language is communicated on top of
Internet protocols and is regulated by a
protocol framework and go a theorem this
is called dev p2p swarm nodes uses this
language to find their most advantageous
place in the network and to figure out
who to share content with but content
doesn't necessarily just have to mean
files a message for example is just as
much worthy of the term thus comes the
idea if we can use swarm to send bits
pieces of file surround why can't we use
it to send messages as well now wait a
minute you might say there is already a
messaging platform in aetherium and
that's called whisper and this is true
but the mission of whisper is somewhat
different from PSS its focus is to serve
the need of total anonymity and to
secure freedom of expression at cost of
performance
now since the swarm network is routed it
knows that the shortest way to the final
destination is and therefore PSS can
deliver messages fast and with a minimum
of network load simply it serves a
different purpose so this is what PSS is
it's a postal service over swarm now
this sense sounds easy right if you want
to send a message to someone you just
slap an address on it and away it goes
well it's not quite that easy
since messages potentially pass through
unknown nodes to get where they're
supposed to be
they should be encrypted so that people
can't snoop on the contents that means
that the actual definition of a
recipient in PSS is not only the actual
address of the node but who can actually
decrypt the message actually the node
address in the end doesn't unambiguously
define a recipient yes to route the
message service swarm using PSS you pass
it to them you pass it the message you
want to send and an address to send to
but let's say you don't want to tell
everybody on the network who you're
actually messaging with now this is easy
just disclose a part of the address
swarm routing will still route the
message the best it can from the
knowledge it has so it will land in some
neighborhood of nodes closer to the
recipient than you and ask before
whoever in that neighborhood can decrypt
the message history recipient or
recipients doesn't necessarily have to
be only one now whatever we don't give
address at all this is analogous to what
whisperer does and the consequence is
that all messages gets passed to all
nodes by all nodes
this has nothing to do with routing at
all obviously but it might be in handy
in some stages of the communication like
when you send a broadcast at probe to
enable a pair to send you its address
but encrypted or to get some temporary
encrypted keys that can be discarded
later now to make life even harder for
screw toe files PSS also employs a
redundant routing and this means that
when a message enters into the
neighbourhood of the destination the
node that receives the message for
forwarding will pass on the messages to
more than one peer this happens even if
the node can successfully decrypt the
message this one anyone trying to trace
a message path through the network will
have difficulty finding out where the
message actually stopped that is to say
the node not passing the message anymore
are not guaranteed to be the one the
message was actually for in fact most
likely it won't be so all good and well
you got the message you can encrypt it
what do you do with it well for this PSS
has the notion of topics all messages
that are sent belong to a specific part
topic which could be any four byte value
but intended to be the first four bytes
of the hash of an arbitrary value PSS
provides a registration method method to
attach code to these topics therefore
when a node gets a message with a
certain topic on it it knows exactly
what to do with it the topic structure
is the same topic structure as whisper
uses Forsyth's envelopes envelopes here
meaning the entity that encapsulates
contents in the sshh protocol in fact
PSS actually cheekily steals this
whisper envelope to wrap messages in
reality all PSS does is just to slap an
address on top of it or not the whisper
envelope also provides an expiry
parameter this is used by PSS to stop
messages in from circulating in the
network perpetually and finally PSS uses
whisper also has a back-end for
encryption and handling of keys as
whisper psst cryptography as well as a
symmetric arbitrary symmetric encryption
so any 32-byte value is actually valid
as a symmetric key and PSS and can be
set at the developers discretion so to
sum up to send a message in a to a PSS
note all you need is a key the topic and
an address or not so what exactly right
now this PSS implement the encryption
has said luminosity control which is a
fancy word for deciding how many address
bytes who disclose we have a generic
handshake module which gives you a
simple diffie-hellman exchange of keys
in case you're too lazy to make your own
and the keys generated by the handshake
module are valid only for a certain
number of messages that is to say
they're ephemeral handshakes can be
optionally also left out of the
compilation of go etherion there is an
API of course available over IPC and
WebSockets for key handling message
passing handshakes and common supporting
functions that are needed a simple flood
protection all nodes that get the
message store the message on swarm and
use the hash of the message or the chunk
to determine if they've seen this
message just before and can optionally
have not forward it and finally you can
actually use PSS with the dev p2p
protocol structure that exists already
the logic so this means you can reuse
Stephie TP code that's already there
this can be either integrated into the
code of the node itself or the results
to a module that enables you to use to
do this in a separate process using
WebSockets as the transport layer now I
probably overstayed my time I'm not sure
but anyway there is a more thorough
walkthrough of PSS on Saturday in the
breakout session quarter past two in
this go through a bit more
through a bit more of the internals what
makes PSS tick the API will present that
and there will also be a demonstration
of a chat app that's actually using PSS
right now I mean it's not a production
ready thing as a demonstration prototype
so and this is not only demonstration
from the screen but actually the
audience will be you guys will be able
to
install the app and chat with us live
air photos well that is to say yes so if
you can't wait till then there are some
links on top is the branch where the
active PSS code is now PSS will be a
part of POC 3 for swarm so this means
you won't find it in the ethereal master
repo now it'll be later on
there is a description of the API and
also there are some code examples for
both the generator generate p2p
programming in Goa theorem along with a
bunch of PSS examples on the bottom link
there and I think that was it yes Donnie
what a good scene hello everyone I'm
going to talk about the first swarm
based application that is aimed at a
wide audience basically the general
public so this is a cloud service that
allows people to share files between one
another and also to sync files between
different devices so this is a fairly
this is expected to be a fairly popular
use case and what we hope to get out of
it is that we are going to see how swarm
actually performs at scale so we are
really hoping to get extensive analytics
and user feedback out of it and also
eventually it will allow us the swarm
team but also the etherium project in
general to sort of use whatever
infrastructure we have built in order to
host our documentation our source code
our web pages and so on in a
decentralized and censorship resistant
fashion and in particular
I was always baffled by the fact that
little store waltz has given us a
wonderful distributed source control
system get and then everybody started
using github which is a centralized
service it's a great service very happy
with it but but it's centralized and
there's already work going on for a
module for get on top of swarm Alex
barracks RC is working on it so
hopefully in the not too distant future
we're going to be able to host all the
sources of aetherium on swarm and have
the development repository basically be
stored in a distributed fashion the
first thing that we did you know for
this application was a GUI based file
manager it was released last September
it's over a year ago it is itself a
swarm hosted web-based
distributed application and it allows
people to manage folders and files in a
very familiar and intuitive fashion
it also has pluggable pre viewers for
various types of content so you can view
images movies and later perhaps PDF and
whatever people can think about so
that's the first thing that we have
built then Rahul Mohammed has in this
summer released a fuse module which
means that you can actually mount a
swarm volume as a directory into your
file system if you're using a POSIX
compatible file system so it works on
linux and mac know in the support yet
it already has read and write support
you can already mount Els registered
volumes and there's one thing missing
which is sort of a safe - ENS kind of
functionality where you can update the
Els resolver with the new content
after you have modified the content of
the swarm volume that you have mounted
so this is also available for download
and you can you're also very welcome to
play with it this is a big one so
currently all swarm content is
unencrypted and in the open and this of
course makes it completely unsuited for
any serious use but and this is what I
am currently working on privacy and
security encryption so here you need to
understand that this fundamentally
differs from a traditional cloud hosted
server client application because
obviously you cannot force people like
not to share certain information so
access control needs to be implemented
in a different way
so essentially right control so who is
allowed to modify something is
equivalent to the permission to update
the root hash in ENS so it is something
that is controlled by a smart contract a
smart contract is what takes care of
write permissions read permissions are
even trickier because you know where
swarm content ends up it cannot be
controlled it just goes all over in the
network you cannot hide swarm content
you can only encrypt it so what you can
do is you can encrypt swarm content and
read permission
means the ability to decrypt it and if
you want to have access control lists so
you want a large piece of data to be
readable for some parties but not others
then what you can do is you can take the
symmetric key with which this data is
encrypted and encrypt that symmetric key
with a number of public keys of the
parties that have reaper mission and use
that as a access control list and this
obviously affects how content can be
referenced
so in unencrypted Swarm the standard
reference to content is the swarm hash
of the manifest in encrypted swarm in
addition to the root hash you also need
the decryption key of the route how this
encryption works so as Aaron has already
told before me swarm chunks of content
into small pieces and organizes them
into a miracle tree and in swarm every
chunk is going to be encrypted with a
unique key and each edge in the
encrypted Merkle tree contains not only
the hash but also the decryption key for
that particular chunk which means that
if you reveal a key at some point in
this tree so one of the Merkle trees
nodes key gets compromised it means that
only the subtree under it is compromised
which is kind of a stopgap measure for
catastrophic failures against losing
keys and also it allows different
directories to be shared among different
parties as a sub directory of different
directory structures and so I would like
to invite Aaron back to
talk about the network testing framework
that we have built yeah hello again
I'm filling in for Victor our team lead
who is here but not able to speak right
now so I want to talk a little bit about
the network testing well the full title
is the network simulation and testing
framework and I have to admit at this
point I'm not the right person to talk
you through this diagram in detail but
that means I'm the right person to give
you an easy-to-understand high-level
overview of what's going on so I'm gonna
start from the right and talk my way
over to the left in the top right what
you see is our network visualizer it is
it's a web-based or a browser-based view
of a swarm network so all those dots in
there would be swarm nodes and
connections between swarm nodes this
interface allows us to query any swarm
node what's its current status stored
chunks every connection the swap
connection was to swap balance what are
the payments and so on so it gives us a
very great overview of a swarm network
of a test network to see what's going on
and it also allows us to input changes
we can click on a node and tell it to go
offline tell it to initiate a data
retrieval whatever we want that would
take us into events so these events we
can fill in to network so simulating
activity on the network telling nodes to
drop offline come online do various
things and these events get packaged
into whole scenarios so we can write a
network scenario of nodes coming online
sharing content downloading content
paying each other refusing to page other
whatever we want and we can observe how
the network behaves like what did what
the network does what the emergent
behavior is in the backend which is
further towards left
we can either simulate the swarm nodes
in process very quickly and or we could
launch a bunch of Dockers with various
so we could host a cluster of swarm
nodes
and and actually run them in full or we
can simulate them so these scenarios are
portable so separating out what we want
to happen on the network and the level
of simulation whether we just want to
simulate the process or actually you
know there's also also one to rather
like a back-end to our cluster where we
can have our whole cluster of machines
running a whole bunch of swarm nodes
doing running through these scenarios
and the reason why we're so excited
about this or why we're developing this
is to test emergent behavior of the
network so normally when we talk about
testing a client you're running code
tests to see whether a swarm node
behaves the way you expected to but a
lot of the intricacies the subtleties of
swarm arise from the interaction of the
different nodes in the network the
emergent behavior like how data is
distributed one node requests data the
other ones forward the request the data
has found handed back all the nodes have
to do the accounting and if you want to
see how this network behaves we really
have to simulate a full network and test
the network so this testing framework
allows us to do protocol testing you
write a scenario what you want to happen
one node uploads content and then what
you expect to happen you expect the data
to be handed from node to node stored
here you know and all that kind of stuff
so it's more than just testing an
individual node is testing that all the
parts work together and this framework
is not is about you know it allows you
to test PSS storage retrieval the entire
you know the whole package and that's
all I'm gonna say about this today will
be give more about all of this on
Saturday so upcoming stuff what's on the
roadmap so one of the things that's
changing is the hashing algorithm
earlier I said when you have a chunk of
data you hash it and that is its ID
we're changing that to actually make the
ID of a chunk be the merkel root of a
Merkel tree that we make out of the
chunks out of the 32 byte segments and
this allows far more fine-grained
resolution
proof of inclusion and and proofs of
custody this is where that comes in then
the syncing protocol has been rewritten
the syncing protocol is the protocol
that governs how swarm nodes share data
how the chunks make it to their
destination and that's the one that was
causing us the most headaches in terms
of performance on the test network and
why if you were playing around with that
you got so many four or four errors
where files weren't found because the
data wasn't being passed all in the way
it was supposed to so that's been
rewritten and it will be merged soon
indeed the entire network layer and the
connection management has also been
rewritten and is ready to be merged and
these changes are breaking so that
hashes need to change the ENS entries
would need to change and in fact we're
gonna have a complete reset of the swarm
test net soon once these things are
merged and then hopefully we'll have a
formal performance test network to play
with
from then on yeah so ongoing work
encryption erasure coating plausible
deniability that is Donny's he talked
about that that's what he's working on
PSS under active development and yeah
the contracts for storage insurance we
didn't talk much about today but they
were in our original orange papers and
they are being developed and tested no
they haven't been deployed yet all we
have so far is contract for for
retrieval and like the bandwidth
incentives but the storage incentive
part from the orange papers is being
developed to and then the testing
framework is undergoing active
development and then also on the roadmap
I want to highlight a few things a light
swarm node we've been contacted by
people who doing mobile development
saying we cannot assume that every swarm
node will want to store a bunch of data
and serve it and earn revenue some
people just want to join the network and
only consume very limited data that they
want you know if you have limited
bandwidth so we talked about various
light modes of operation nodes without
storage or nodes without a lot of
bandwidth
so we've been developing the theoretical
framework how those four nodes should
fit into the network and we intend to
develop those to some various light
modes of operation to make it easier for
mobile science and other light lines the
payment infrastructure as I alluded to
earlier is being developed to be far
more generic and to allow for four
different services than data storage and
retrieval one of them being you know
communications broadcasts and other
things service networks but they will
talk about on Saturday and Victor will
also be talking about distributed
database services and how they fit into
both the swarm network itself and the
incentive layer so that's that the
update from us our team is Victor Donny
Fabia who and Anton from the foundation
and from Jack we've had a not just one
but two Lewis's giving us great a lot of
time and code so big thanks to them and
this forum team is still growing we're
very active development and adding
people yeah so there's even more coming
and so I want to make special
advertisement to our breakout session on
Saturday the session on p2p technologies
blockchains in data and communication
there'll be updates from the swarm team
will talk about swaps where and swindle
will talk about distributed databases
there'll be a PSS update there'll be a
interactive chat demo and updates from
other teams live pier will be talking
about streaming services of a swarms a
lot of good stuff so we hope to see
there and with that I'm going to end a
little bit early but thank you for your
attention
[Music]