then I'm the CTO at bit go and so Big O
is like an exchange API that many
exchanges like this then call bit and a
bit used for their wallets so we you
know you do scalability and security oh
yeah my name is Jake Craig also work at
coinbase senior software engineer on the
the crypto payments engineering team so
we're the team that sort of runs all the
blockchain integrations at coinbase
if you do sends or receives hot wallet
cold storage all that kind of stuff goes
through our systems yep I've been
working in for business
all right on the Kuna cool and I've
probably introduced myself as well so
I'm Eric Scribner and I'm also on the
crypto payments team with Jake and
coinbase and did a pretty good job
summarizing what we do their stuff I'll
leave it at that and all right so we'll
start with the we'll start with some
softball questions here all right so
this one is one of my favorite ones I
think this should be asked more to
theorem and DEF CON events in general so
if you were an EVM op code which Evo mob
code would you be and why
well I'm on stage in 2018 so I think I
have to be the PC opcode all right all
right I can see that oh yeah and the why
why yeah what do you enjoy that call
because I think it's what okay all right
fair enough and Jake which one are you
gonna be so good I've got two I think
the first one you might be what you're
guessing there is a NOC code called
coinbase which is a very convenient one
but that's not gonna be my choice my
choice is gonna be X or which it's just
cuz you know all the new cryptography
stuff in coin bit coin base in the
community coming out and stuff it's just
XOR with some powers basically all that
at some level and it's just a really
exciting thing to play with and it's
this very simple thing but it's just so
powerful all right know what no one
picks self-destructs okay that's good
that's yours
mine okay so let's kick things off let's
move let's move under the medium ones
here okay so I really like this first
one so what is the what is your biggest
challenge integrating with the theory
I'm at this moment I'll start with with
you okay when huge challenge is a Nantes
management in terms of you can have
rapidly fluctuating gas prices across
the network and if one transaction gets
stuck then it slows up all the others
and you either have to resend the first
one or just wait it out that's a huge
challenge yeah I think we see that as
well it's a pretty common issue when you
have very large wallets so with you know
with large wallets and when in aetherium
as many of you know the accounts have a
nonce and whenever you create a new
transaction it has to have a nonce which
is monotonically increasing and the way
that it was designed that way is because
it's a lot easier to track doubles
protection in all that and so say you
know you you're running in exchange or
something like that and you're gonna be
sending you know ten transactions for
five minutes and this this can be you
know 100 transactions you know over 50
minutes and if a transaction number
number 4 you put in a wrong gas price
then by transaction 50 all your
transactions can clear because 4 isn't
clear so I think that that's a big
that's a big challenge there you know
you want to talk about some solutions
yeah yeah like you said we've all sort
of had this issue and it is one that
tends to come up at scale as you're
doing just a lot of them back-to-back
right if you're an individual wallet you
know you send one transaction if it
takes a while maybe that's okay or you
can really easily sort of replace that
it's just sort of replaced by fee
mechanism and it's an interesting one
because like it sort of depends how you
structure your hot wallet as well
they're different ways to do it some
exchanges you know tend to use like a
sort of single or a few addresses for
everything so they have a lot of
transactions going through that address
other ones use you know contracts and
stuff or you know lots of them like we
like coinbase in our wallet we just
basically have just you know hundreds of
millions of addresses and so we're
always have it all over the place and so
when we hit the situation it can be
fairly limited in scope but it can
affect like a subset of customers and
there's not great solutions for it yet
other than replacing it which is a
solution but it's a more sort of
complicated one to manage and especially
when you've got a lot of these going on
and so this this thing where so the gas
prices can spike and sort of delay
everything is really challenging this
example that I've seen in our systems
before is a transaction two minutes
apart same gas price one took two
minutes to confirm in the electric 13
hours and that's not a really great way
to build you know systems interior to
build a business on the theorem and so
it's a it doesn't just affect us I could
just affect a business doing this who
doesn't have you know as sophisticated
infrastructure to handle and and one of
the things that we've sort of talked
about and seen come up with a few times
is you know this idea that comes from
sort of Bitcoin of this child pays for
parent style of transaction where it's a
similar mechanism of sort of recovering
sort of pending ones that are like
chained together in some ways like in
this case it's chained by like that
nonce it's different in Bitcoin but the
theory is sort of you you put a new
transaction out like on top of all these
with a high fee and then that sort of
can subsidize the rest of the lower fees
and get it back to closer to what the
real network fee is so miners will pick
it up and look up all of them at once so
you just send this like transaction with
this really high fee and then all a
sudden you know you're maybe you know 50
whatever transactions are all all can
get confirmed and you can sort of
recover that and have a sort of a better
experience for your customers and so
that's yeah that's that's an interesting
one yeah so let's talk a bit about that
so so firstly you know RBF like to
replace by fee it works today because
it's you know you can do it yourself but
one of the larger I think confusions
with that you know when you change the
Pheo so change what you're signing and
therefore the th ash gets changed so
you're displaying a TX hash TX ID to the
customer of the person receiving the
funds then it would be different
with CP FP it's you know well-known in
Bitcoin land the the common challenges
you know the idea is you have ten
transactions not confirmed so you put
another one transaction 11 and it's got
an outsized fee and the outsized fee
helps to tell the maya that hey you know
if you want to get this outsized fee you
you you go to mine all these other ten
first um the the challenge for the miner
then is how are they going to figure out
that this string of ten transactions
combined with xi is really worthwhile
for them to mine because if they were to
do this infinitely like after a thousand
and thousand and one transactions then
they can run out of memory just waiting
for you to come up with another big no
that was the thing that's brought up it
came up in if you're in maybe a few
months ago now when there was this
exchange that was sort of incentivizing
lots of volume on the network so it was
just kind of backing up the network I
think this is another sort of big
challenge with the theorem is that
individual token sales individual just
knew you know games or whatever can like
significantly impact the network very
very quickly and very fast they start
the token sale the fees skyrocket now
everybody in the networks paying more
any app you've built is just now all of
a sudden paying more and then now you
also as you know anybody sending
transactions get them stuck because the
the estimate can only update once it has
all the new data so you're kind of at
this you can't ever predict it
necessarily you can have some sort of
models and like generally it works and
it updates fairly quickly but there is
this sort of delay and that was a thing
brought up when I was started we talked
about in theory I'm a little bit I
believe on the parity repo is an issue
that has some sort of proposed model but
they're specifically talking about sort
of like toss vectors and stuff with this
how long you track it what's this buffer
and is that configurable and how does
that actually work so so to summarize
here it sounds like it's like volatility
of fees is the biggest challenge here
and there's no good solution at this
point is that the is that the deal
there's a solution it is a challenging
one I think because like I've been said
here the you know transaction that's
changing and stuff it's a more
complicated thing to like re like match
things up if you you know have sort of
two pending trends
action for the network and one of them
will confirm and generally it would be
the higher fee one but like I think in
theory like the lower one can also
confirm so now you have to have these
two sort of states and like which
transaction is real and then you know
one of them will confirm and so it's
just more complicated problems
employment has become a potential
challenge to because it's even it's good
potentially ok so now we have a new kind
of talks on this bend where maybe miners
run out of run out of memory space when
they're waiting for all these pending
transactions maybe but if if
hypothetically that did exist if it was
implemented poorly it could lead to this
new design pattern where everyone's
putting no 1g way or whatever it be as
all of their transaction prices and then
if it doesn't clear within some X time
period then they put out a transaction
to kind of clear those out so we have
this now almost waves of transactions
going through just because people want
to pay lower their costs right it's a
tricky situation
all right ok well it's it's so all three
of you seem to have consensus yeah one
of the things right I think another one
that it's not just us but and we
definitely experience this is just
running notes very running nose keeping
them up-to-date on the size of the
backups and stuff in particular these
sort of like archive state notes that in
a store like the full history I think
offeree of you know parity just released
a website about like that constantly
tracks the size and so if you run like
parity you can get it to like 200
something gigabytes but you don't have
this sort of full history one and that
one tends to be I think around these
days 1.8 terabytes or something and so
just the size is the thing I'm just
transferring that over the internet when
you need to like to spin up a new one
just takes a lot of time I think ours
are ones when you spend them off for
like three four hours right now to sort
of do that we've got things to get them
faster but a fresh thing is like I think
the last one I did something like that
at which which kind of flows nicely into
the next question which is do you do you
think that integrating with the theorem
is getting harder at that low level that
we're talking about here and if so do
you think that we're at some sort of
critical juncture with respect to that
I think for for instance I mean the
blockchain size keeps increasing
everyday so for that purpose it is
getting harder in terms of resyncing
nodes or syncing a new node if one
crashes or something like that you lose
a database in a lot of ways though
there's a lot of tooling that makes it a
lot easier to integrate things so things
like truffle or ganache are becoming
really really easy to use and there's a
great user experience around dev
workflow now for even just for local
development but also for production and
it's it's this interesting kind of
duality so some parts are getting harder
in my opinion in some parts of getting
easier potentially I think to go to
production with a theorem is getting a
little bit harder I really like the
developer on-ramp so you know the
developer tooling and all the you know
building on tools are getting better and
better welcoming new new new developers
and that's why we have so many
developers at this conference I think
that so far is running a node when I
last fired up for myself you know we had
lots of nodes in production but
sometimes you want to hack yourself I
think death if you you know running in
the fast fast mode it's actually not too
bad and I think it's right now it's not
a critical juncture juncture because at
this point the the block processing time
is still less than the time it takes to
propagate and it's time that one you
know get stopped and that you know we
can talk a bit about more about that
like that was different earlier this
year but I think that right now it's
quick where you can fast think all the
way to the hit of the chain and then you
can just you know your blocks are
getting pushed that's quick enough with
you know modern hardware it's not
totally a problem and of course you know
you really want to be running a node in
in production and running several of
them so that's a different challenge but
to get on board I think it's actually
still pretty easy either like onboarding
definitely doesn't having a lot there's
a lot of really cool stuff there to like
get up to speed and like start
able to do things but like you said sort
of at this sort of like scale thing or
when you're running like businesses that
are worth a lot of money and a lot of
money's moving through you also want to
like have sort of the full history
yourself and always have a copy of that
and so you this kind of initial sync
time Wow like I think for a lot of these
cases this is not particularly important
or like syncing sort of every block into
sort of your own database that you have
sort of your own full history that you
can analyze outside of you know just the
sort of latest state like one of the
things we have is our data team once you
know full copy of the blockchain in you
know sequel that they can sort of
analyze and do things with and right now
so we sort of manually build tools that
do that kind of stuff and just takes a
while right just because of the pure
size do you know how big that is how
much data that depends on how you want
to index it right so I think that's this
project I saw it part of the grant team
showcase it was called vulcanized Evie
or something like that
it was pretty cool you told them what
you want to index and they they will run
it through and put it into a post Chris
I think it's a great idea and it's nice
it's lowering the bar so that not
everyone has to go in the back so you
know you guys probably blew in an exit
we have an intensive so everyone's
building one but the more it can be open
stuff all right I've got it I've got
another easier one for you well maybe
it's not easy actually it could be easy
all the way to hard so so parity again
or a lot if you like that one
I'm partial to parity I found it's been
more reliable and faster sync has been
nicer and we can get into the whole
parity trace parsing issue we will yeah
but we can definitely we can definitely
hop into that yeah that's definitely one
a bandage parity had early on yeah well
that's what I've heard it is with parity
what we're sort of ending out here is
this idea if you're only terse can you
see contracts and heathyr around they
call them internal transactions and sort
of detecting those and seeing like where
money moves is a complicated thing I
actually give a talk on this yesterday
that it'll be a recording of and stuff
about sort of the ins and outs of how
that works and stuff but from my
understanding I wasn't
just happened but parody was one of the
first ones to give an API that made that
more like accessible and easy to sort of
figure out what happened and guess now I
believe since 1.8 it was does have an
API for this - but it's not a
standardized API it'sit's actually a
custom one for each one and so there's
no standard way to process these and
that's setting something that would be
really interesting to see in the future
yeah it's it's interesting because like
there there are lots of transactions
that happen that appear nowhere in the
blockchain right and so I think that's
one of the interesting differences
between aetherium and a lot of other
ones but Ben what about you yeah so the
first question I think both parody and
get ok random both depending on what the
network situation is sometimes some of
them a challenge is processing certain
types of transactions and blocks and
some of them you know then the other one
goes faster than so I think running both
this is the way to go with I think that
this is a huge topic internal
transactions for us it's one of the
largest pain points so a bit go we use
multi-sig wallets these are all the
contracts in smart contracts and for us
when when we send out from the smart
contracts we want the receiver to be
able to see the funds on more modern
exchanges they do you know record the
deposit because they can look at these
internal transactions and so a little
bit about international sections so in
aetherium you have transactions and
transactions are what I think are the
first we can call it like the first hop
in which an account send something to an
address where the address is either
contract or another account and a
challenge is after that if you send it
to a contract and tell the contract to
do something then internally the
contract is is making changes to state
and doing internal messages some of
which may be sending funds like a
theorem to other dresses now the
challenge with that is that currently in
the web 3 API there is a get transaction
which does the first one it shows you
all transactions that come you know from
accounts
to smart contracts but usually what a
multi-sig wallet is is that you would
send a transaction from one of the sinus
to the multi-sig wallet and then a
multi-state wallet would then send out
the funds from the smart contract and
that is a smart contract transaction or
internal transaction so there's no you
know RPC or API to easily get ahold of
those transactions as a result some of
some of the exchanges they don't
recognize deposits automatically for
such or such not contracts in and this I
think will be further accelerated
because now we're moving towards oh is
that unified the unify idea Wallis
versal yeah universal login wallets
which I think I'm a big believer of from
a UX standpoint I remember yeah also
smart contracts so we need to solve this
problem you know as an industry and make
more people actually taking deposits
from small contracts because that's what
even this is about right it's a small
contracts
yeah and it's it's a scary thing to
Parsa process these two you can have
some pretty like dangerous sort of bugs
that come up from doing this the wrong
way and so there's plenty of exchanges
and stuff who don't want to do this yet
because it is this sort of scary thing
to get wrong whereas like transactions
and stuff are fairly straightforward to
kind of tell what's going on these
aren't so it's a big risk and especially
as we move to more and more sort of
wallets doing things these are gonna
have to be supported and me to make that
easier and that kind of flows nicely
actually into the next one that I kind
of wanted to throw out there in less
sorry we're gonna go okay yeah so so
alright so I guess the next question
would be wallet contract or like scan
individual blocks and transactions and
invoke transactions which one which one
do you prefer
yeah it is there's pros and cons right
so either you're generating a new
address for every deposit you want to
take as an exchange and you have to
monitor these interventions actions very
very carefully and it's theirs then you
have to sweep them into your your hot or
cold wallets and it's this big process
or even
as a contract side of things if you
build things as a contract I think bit
tricks does this and maybe some other
ones too
things like sweeping gets easier you can
monitor events in your smart contract so
internal transactions aren't a problem
anymore
but now you have increased cost for each
individual customer who needs to a
deposit address but may never deposit
any ether in there you have to handle
tokens as a completely separate case it
definitely complicates things and there
is that trade-off one of the things I
don't particularly like about wallet
contracts especially if it's like a
single one is that it sort of it puts
all your transaction history in one
place privacy is a really important
thing and we want to have and so
contracts especially the sitting because
of the cost like you're going to be
incentivized as any sort of business to
do sort of one right unless there's some
extra cost or something for you know a
new address and if you have one now all
of your history to that exchange is in
one place and so anybody wants to see
how much ether you had added how much
you sent there they cannot tell that it
may be how much you have in that
exchange maybe that makes them more of a
target if they follow a pattern like
things like the trucks as contracts like
you can detect them easily and so you
can easily see how much of it checks is
receiving through that pattern or is
like the way that we do it which has
some other downsides and cost is that we
generate new addresses just all the time
almost anytime you get an address from
us it's a brand new one that's never
been seen before if we send transactions
it's coming from a new mattress it's
never been seen before
ok that has some things of just having a
lot of addresses its own problem and I
think we might get it up later to about
you know sort of batching transactions
and stuff and doing a lot of sins you
have some interesting challenges there
in certain transactions but I think like
I don't like this idea of just like
linking everything together all of your
activity in one place especially if
you're gonna like send from there too so
I think in terms of having one address
client
I mean per customer exchange pretty much
many exchanges are used to this concept
from you know a variety of other digital
currencies and so I think on that matter
what we're doing is whether or not
you're a contract or address you gotta
have one one address you know one one
address per downline
you know customer deposit a dress and so
what happens in the end is that when you
try and do that with contracts you are
you're probably going to be using some
kind of forward or contract in which
what it is is a really small shim that
forwards the money from this unique
deposit address for each customer into
some some main and you know sometimes a
challenge with that is is with with us
at least we wanted to be a multi sig
platform and so we wanted you know all
all our wallets and contracts to be you
know not controlled by a single key and
that was one of the requirements that we
set out to fulfill and that's why we did
it that way I think that I mean one one
one advantage of that for for us that I
guess I haven't talked about is that
with with contracts you're you know
instead of looking at the internal
transactions by yourself by looking at
transaction receipts you can look at the
events because you can get contracts to
file events and and actually you can do
it within the 21,000 guests with it
you can get one event or something like
that badly enough and then you can you
can code your indexes to pick up those
those events and use them for you know
deposits so so there is one advantage
there the disadvantage is you know pay
you have to pay the gas to to deploy the
contract which which can be very costly
well I mean I think there are some IPS
that are out there not too far away from
being minded that will allow us to do
you know basically paid to script - in
other words you can pay to an address
which is a contract address without
actually it having been deployed yet and
then someone else can then deploy after
they see that okay someone actually
deposited there and this is really
useful in the case of when exchange that
has a hundred thousand users and only
fifty thousand may use s so you only
need to set up the address after they
doing there I think about like the gas
limit stuff right so this is the thing
that different exchanges have to make
different decisions on of customers sort
of sending aetherium somewhere and if it
might go to a smart contract
not in the trade-off of like what sort
of UI do you present there what sort of
gas do you charge your users because
your standard aetherium transactions a
flat 21,000 you can sort of guarantee
that or decent a contracts there's no
guarantees necessarily there's things
that you can estimate it and it's
probably right but it might change to
there's some ways it could and like
that's an interesting thing in exchange
you have to do like us on our sort of
brokerage of just corn-based comms
you don't see any gas stuff when you
send and it's a very intentional
decision to keep it simple because it's
a very confusing UX sort of design
pattern it as a sir people have talked
about in the design tracks here and it's
this interesting trade-off of like how
to handle that and I was like a business
how do you set that and how do you sort
of expose that to users yeah it's an
interesting question for exchanges and I
mean coinbase you kind of both but but
exchanges and wallets really same UX
problem really it's the same UX problem
that aetherium and really all the
cryptocurrency industries any other we
do right now it's just it's probably not
just a theorem and but we can't really
just sweep it under the rug I mean the
the the main issue I think really is
that customers expect them to pay you
know they're used to paying fees which
are percentage or a flat fee and in in a
theorem and other chains you're really
paying for compute or space and
interesting all right so I think Jake's
you'd mentioned batching briefly down
there you guys want me chat about that
of it what do you think about it is it
something that the theory needs is there
some sort of alternative thing that
maybe would also be interesting to have
yeah I think I only first brought up we
were sort of talking yesterday and I
sort of brought this up and I hadn't
thought too much about it deeply but
like I mentioned we have a lot of
addresses and so we have all sorts of
customers setting and all sorts of
amounts so you know we'll end up with a
lot of addresses with very small amounts
of ether I will need to so let's say we
have a set of addresses that up you know
one eath in each and we need to do you
know attend etherium sent well we need
to sort of combine those into some other
address and then after that confirms on
chain all you know ten transactions that
might take what we need so we need to
first send ten wait for them all to
confirm then once they confirm we can
then send out this tiny three so this
we call this like consolidation
internally and this is sort of a
complicated thing to manage and it's an
interesting thing because like Bitcoin
and sort of these you TXO chains they
have a different way of approaching this
that doesn't have this problem and so
once you in it one of the reasons they
do that is so you can have sort of a new
address for every sort of change amount
and so you can basically always be using
new things whereas like on a theorem
it's really hard and it's all about this
privacy stuff again where you wouldn't
want to do that because if you wanted to
do a cent you'd have to do all these
like intermediate sentence to finally be
able to do it and so I think it'd be
interesting to consider some sort of
like batching strategy or like multi
transaction thing where you could say
you know from all these addresses I'll
include all the signatures for them do
all these transactions at once into some
like destination and as sort of one
atomic send there because also you
probably don't want to do because you
could also I guess another way it's sort
of consolidating and then do a good
thing at any send you could do like ten
individual any sense that destination
but if it's like a payment service they
probably aren't gonna handle that well
if they're expecting ten even they get
like ten twenty sends they're not gonna
know where it's supposed to go to right
pretty bad UX if a user goes to send one
transaction and they received ten on the
other end yeah that's probably confused
I think with the theorem actually it's a
bit it's pretty cool in different sense
that in Bitcoin you know that you have
these inspects and you can tell whether
each of the spent is economical to spend
or not in a theorem you can almost say
like okay if you've got like some dust
you can just wait because you're not
spending part spend you're spending from
the account so you know if someone sent
you a lot of you know one one ways you
could wait for I don't know resilient of
them before you actually took it out
usually I think you know consolidation
is it's a good idea to just do it like
when the account went when the network
is less loaded so you know when nobody's
holding any I SEOs on weekends or
something like that when the gas price
is pretty low and keep it low so that
you don't actually impact the network
because you very well could one of the
interesting things I want to ask both of
you here
you know what have you seen in regards
to batching out outgoing transactions so
you know with with with other with other
coins without getting too much into that
a common pattern is you wait every five
minutes or ten minutes and you batch out
a send you know of whoever else is
trying to withdraw during that time with
the theorem this is largely not been
done yet even though it is very possible
even today without making any changes
whatsoever to consensus at all
you know just use a smart contract and
do a multiset yeah for me and like some
of the conversations we've had about
this sort of thing on aetherium and
Bitcoin really any chain is this is
privacy again because like you now link
all your sort of customers together in
some ways and we want to generally
prefer that like we like to sort of
default to privacy and try and maintain
that as long as we can and so when sort
of batching comes up and that that's
always the thing that it like immature
like wallet contracts like putting
everything in one place and maybe just
the theorems the wrong chain for the
types of things that that I'm talking
about it I think there's ways we can get
there but for me I haven't seen it and
you can definitely pull it off like you
saw the smart contracts and stuff I
think that's kind of the cerium for lack
of a better word purest answer to like
the bathroom question is when you just
use a smart contract for that and that's
kind of aetherium and for everything
which isn't a bad thing but maybe
there's trade-offs that you don't want
to make there in your in your
server-side architecture that we so
you're saying earlier if you have a
wallet contract I guess it becomes very
convenient to do these kinds of multi
since but if you don't then there's no
like alternative mechanism right yeah
you could theoretically design a
function calling your contract that
would multi send to a list of addresses
that you pass it okay yeah and it's all
technically the same aetherium address
but then it just happen all internal
transactions but then like you said
they're all linked together and well you
also need a gallant need a balance on
that account too that's like sort of
public and probably associated with you
in some way so like you like its offense
valued structure well it but like you
don't want to probably have all your
money in one place maybe but the thing I
think essentially lots of I mean lots of
exchanges end up with a hot wallet so
and and I think what shape-shift it
doesn't matter as much because you do
like one transaction at a time right so
you'd like to send that but maybe one of
the reasons why people don't use this as
much in aetherium is like I think it
theorem people are really used to seeing
a transaction where they have a source
and a destination and if you look at
that like either scan you know it's
quite clear it's always got a source at
a destination so it's very one-to-one
how people like the mental model of how
they think about the transaction you
know in another change you always have
both developers yeah it's also I think a
little less necessary in etherium
because like we talked about that not
stuff earlier in a theorem you can do
these sort of serial but like back to
back sends like on from a single account
right if you just increment the knot so
you can continue to like to send from
the same account so you can you can
queue up all of them at once without any
sort of delay or is like on Bitcoin and
that style of chain you have to sort of
if you have a big sort of input you have
to sort of wait for that to do its thing
and then do another one or there's other
ways you can do it but it's a lot
trickier words I get through I'm just
like just keep sending from the same
account and it'll let you do it and like
it should generally work and it doesn't
mean too much of a problem we've seen or
like any sort of need for batching it
either there's something like gas
savings potentially yeah I think I think
that that like the gas savings um I
think that with the theorem maybe we
aren't as mature with doing gas
optimization you know with Bitcoin for
example because things are so expensive
you wanted to save every little bit you
could so you're paying the same amount
for signatures on the inputs and if you
can send to ten at once instead of one
at a time you do that right but with the
theorem I think it's still a little bit
earlier with the gas savings now it's
mostly like how do we price that way you
know get gas price rather than actual
guest spent and really for all of these
conversations change once like an actual
Caspar sharding some scaling solution
gets implemented they're all just
completely different because we have a
whole new set of problems and yes yeah I
mean our next aim to measure that would
be like with the with the sharded
network could be much more complicated
I think I think we might switch gears
here a little bit so uh we got here I
guess I would love to talk about privacy
with you guys so how are you thinking
about it what do you think the status of
privacy and fungibility is in general
are we there yet what are some of the
interesting prospects and how do you
trade that off with scaling in terms of
aetherium we're definitely not there yet
for privacy okay definitely not there
yet but it's like snarks exist now but
it's it's really not easy to use unless
you have PhD and even then it's it's
difficult it's it's definitely tricky
and really they are there best
implementation should be in some sort of
plasma like implementation or in a shard
of some kind so it's almost a catch-22
and that which which comes first scaling
of the privacy yeah so I think it's not
that not by any I mean you can see how
easy it is to track you know I think
this can they have label like
shape-shift one shows you know that's
that's the most public think of a thing
I've ever seen
having said that I mean I don't feel
like you can really trust so call us you
know pseudonymous nosov bitcoin you can
always almost think about it as those
that those transactions will be review
when you think about privacy it's not
just about like privacy today and okay
maybe it's a little bit hard to track
today but the blockchain is forever
right so so you going back going forward
like five years later all of that data
is gonna be there
so you're gonna have to live with
whatever you're putting on on that today
so I think that's a key you know
security is really important and privacy
is one of you know one of those those
important things for security because it
makes it just that much harder to secure
something if you have a very big target
amongst other reasons so I I just don't
see this as something that is being
worked on in us yet from a theorem I
think that we have scalability
challenges
and I think they're gonna go after those
first because that is what drives
adoption I mean whether you like it or
not they want to drive adoption yeah I'm
really excited for sort of the prospects
of like the future of privacy but yeah I
don't think aetherium is doing maybe
enough to to be concerned about it or to
like there's definitely teams in the
space working on sort of off chain sort
of style solutions which I think could
be really interesting but just the
general sort of blockchain community as
a whole there's a lot of really
interesting stuff going on right you
know there's all the dizzy cash and such
doing all the snarks and then there's
you know starts now and you know bullet
proofs and all sorts of things happening
but I think you brought up really good
point that about like this sort of like
foreverness this sort of infinite sort
of data thing and this has already
happened with Manero where you know they
had some sort of bug where you could in
theory sort of I think in some ways you
could I don't know the details but you
could effectively like sort of D
anonymize some of their transaction
history because of some sort of you know
vulnerability there and that's sort of
already happened and so in a lot of
those new cryptography it's also new and
you know most of your target for these
things like fairly like well-established
tested and stuff and so we do these you
know security audits but a lot of it's
also still backed by this this there's a
word for it but this idea of like it
hasn't been broken yet and a lot of
people have tried we can't prove it's
totally secure but we think it's pretty
good and so we'll keep using it until
then and a lot of the newer stuff Reyes
imagine would have like similar
properties except it doesn't have like
it's not battle tested in that way and
so while there's like really smart
people doing it in developing this stuff
I think that's a really like big sort of
risk in general to it but I think it's
definitely a worthy goal that we should
we should go for as you know businesses
and anybody wanting to build on this you
know I don't want my whole history to be
for everybody to read anytime I send
somebody something from any of my you
know aetherium wallets they can
potentially see my whole transaction
history and I could split it up in all
sorts of addresses and try and do that
but then it's it's like that problems
describe barely even like a just
individual level not even the exchange
level of well that's really complicated
to manage it was expensive just few eyes
cool alright so so another easier
question here but what are you guys most
excited about with respect to your
theorem it's a good question I would
like as a something we would like to see
in there or is it something you would
like to see some project that you think
is interesting or some you know future
prospect are you really looking forward
to I definitely think that for me it it
probably goes back to the whole nonce
thing that we kind of started with it's
that's a one huge pain point and
something some solution to that even at
the protocol level would be very nice
it's also something that Jake mentioned
yesterday when we were chatting was it
was there's it's very interesting
concept of because we have it's more of
a solution to the internal transactions
parsing problems that we've had where
maybe we could implement events at the
protocol level like contract type events
at the protocol level and I hope I
didn't take the one you were gonna say
just now but no no I think yeah yeah
yeah it's a really interesting idea like
this I like because I think it's it's a
good way because like events are this
later known density and people know I
deal with them give the vic-20 and 7:21
and they're gonna even continue to be
used and people have tools and stuff
around processing them in really
interesting ways and so I think that
would be a really nice way that you
could you know extend you could bring
these sort of internal transactions out
of this sort of mysterious world that
you need to parse to get them and gets
that's one way yeah I think it's a great
it's a great idea I mean when you look
at it what what people what a lot of
people actually want is they want to see
when a theorem moves and there are a lot
of ways that smart contracts can send
that today which don't get shown up with
get transaction API so I think events
are one of those ways if you can fire
those events whatever whenever any eff
moves that's that's a big way that that
will make it easier for integrators to
seize these transfers and credit
deposits another way you know could be
give them what they really want which is
again transactions API which actually
gives them the transactions of all F
and you know that's easier said than
done but you know I I don't even know if
it's that hard right like we we've all
sort of experienced this parsing of
these traces like it's a doable thing I
mean it'd be really easy to like write a
tool that does this like it's it's yeah
so much that it's like difficult it's
just that it's just kind of there's all
these nuances that don't seem right be
really dire augmented and so like
various note implications could
absolutely implement these api is
especially hands I have sort of custom
ones but I think would be ideal is like
a standard sort of like our PC API that
they wouldn't require like protocol
levels because I think it's gonna be a
lot harder to sort of sell but you could
absolutely just have it would be a new
kind of RPC API extension to web three
or something like that and I think
people you know someone that that for us
because we care so much about platforms
and having developers use platforms you
know if they have to go for a talk like
yours great talk
but if they have to go for a talk in
order to learn how to do it that's
probably too hot you you want to just
give it to them
right away just give them the API and it
should do anyone more on the topic of
things that I'd love to see nodes
implement is the I believe it already
exists in the EVM as far as I know I
think it came with metropolis is the
returning data with a revert opcode
being able to have actual error codes I
can send back in my errors and not have
to do extremely tedious solidity
debugging for hours to find out my typos
I didn't see at this time I'm sure there
was standardized the return codes yeah
okay yeah I know they implemented in
Metropolis the OP code to actually
return just arbitrary data that I can
pass a string to revert but I haven't
heard anything for a standardization
yeah I think there's someone working on
it I think it's really important it's
like kind of like you have HTTP status
codes you should have returned off
course yeah
yeah one other thing I think is
interesting that would be be cool to see
it doesn't exist but it's is like some
more sort of infrastructure or sort of
tooling around like running notes and
running notes and sort of like a high
availability set up right and a lot of
the nodes of today's sort of expect you
to run them and this single
configuration of just you have one thing
running on one box and you expect it to
never go down and never break and that's
just not reality right these every piece
of software will break at some point it
will go down and so if you're having a
lot of sort of transaction stuff going
through also you can't just use one like
it just wouldn't work at this sort of
scale you may see and so you need more
than one and then you get into all sorts
of issues where you make a request to
one node through a load balancer and
then you make another one and you get a
different result back because it hit a
different node that's a different part
of the blockchain so you have to sort of
deal with this in in like interesting
ways whether or not you sort of take
data from multiple nodes put it on your
own database and sort of handle that
inconsistency there or just run one node
I suppose as an option there's a lot of
choices here but there's not much
information on that and I think other
block chains have all their like
proposed ways of running them and more
of this like high availability set up
for like if you want to build like you
know reliable sort of infrastructure
around these to make sure that you know
nodes aren't your problem and you can
just kind of focus on on the chain I
think we need some better tooling there
yeah I think that sounds like a great
grant project actually I mean think
about it right like cuz we have like I
think a theorem is really cool we have
like the most clients I think of any
blockchain probably most different
implementations you could almost be like
connect to the web three or gasps and
parity and be like a load balancer
crossbows and and only accept things
that go with them agree on or come out
yeah we have an injury yeah yeah we have
like an interesting set of that kind of
does that but not about the agreement
thing where we actually sort of chained
them together we chained together
different node implementations so that
if there is some sort of consensus
breaking rules it'll sort of stop there
and not sort of get to our the rest of
our systems because like these things
happen and this is a way to sort of
mitigate them and so yeah trying to sort
of detect consensus and detects like 51%
attacks and things like that there's
probably a lot of interesting tools that
can be produced out of there that I
think a lot of exchanges in companies
building are
aren't doing on their own but there's no
sort of shared stuff out there that's
actually quite interesting but also
risky so when you look at like consensus
breaks sometimes what people do is they
attack one of the implementations and
cause it to you know maybe see a block
as invalid when it should be valid and
you know the safest way in my opinion is
to connect to both of them and then see
if there's a substantial difference in
the two chain heads by you know a
significant amount of work right if you
change both of them it's actually good
in that you were well kind of good in
that you get the most valid one in that
both of them will see it as that it but
that could be something out there that
has more hash power that you don't see
as that it so right yeah it's sort of
it's sort of like it gives up lightness
right in the event of a chain split it
so it gives up liveness since instead of
like just keep running on and like
hoping you're on the right side of the
chain it says we're just gonna stop and
we're gonna figure out what's going on
here and we're gonna investigate it and
then attempt to make the right decision
there and catch up or whatever it needs
to do that's sort of that's adding a
great point though that is the sort of
trade-off it makes to do something like
that we're gonna stop the world and
until we figure out what's going on yeah
the world weird all right I think I'll
do that I'll do this last question and I
would love to do some some audience
questions if you have them in the last
ten minutes there so we'll do an easy
one which is so what's you so what's
your favorite thing about working with
the theorem versus the other
cryptocurrency is equally obvious answer
smart contracts right there's Stoppers
so much more that you can do so many
more interesting creative solutions to a
lot of the problems that aren't specific
to aetherium necessarily things like
it's has offered its own share of
problems but and and really just a
community like there's a huge community
around the area and building this out
that does make it never a boring day
right never a boring day it's it's
always interesting there's always
something new happening it moves so fast
that you have to almost struggle to keep
up with it sometimes
that's a good problem to have yeah I
think that the community in aetherium is
this for me one of the strong probably
the strongest one it has what the most
developers which is a great indication
for me as an engineer where it's going
to go I think that in general you want
to build when you build something you
want to see like other people are using
it so there is ad option as well and you
you feel the most you feel more welcomed
building something on aetherium it's
like you know here I built this and and
and everyone likes to try and see what
kind of depth you build and I really you
know I really enjoy that yeah I think
that's a big part I think yeah it's
similar answers for me is yeah I mean
that what people can do is smart
contracts like I don't really do much
work with them myself but to see what
people are able to produce especially
like all the recent sort of financial
instruments and stuff things like like
die existing and you know the ability to
be able to create some sort of thing
that has this stable value without
anything actually behind it is this
really interesting concept or all the
people are doing with you know the the
relays and all the variants of you know
loans and all that is somehow happening
in this world and it's kind of crazy
that that happens and you can just very
easily go on to some you know some
wallet app and instantly you know take
out a loan or something and and that
kind of works and then the community is
yeah also really I think generally
healthy community and doesn't have as
much of the sort of tribalism you might
see across all different block chains
there's you know some of it of course as
there is anywhere but generally I find
that the community is very sort of open
and willing to work with other block
chains up and try and push sort of
everything forward I think like one
great example this was the talk
yesterday on verifiable delay functions
that they're talking about doing in
serenity is this the it's this thing
that's not specific to aetherium like
they're actually bringing in other
chains and want to work on this together
in collaboration and come up with this
just better cryptography and better
system to you know have this you know
new concept to work for for everyone and
not just be this like singular thing for
a theory and like it's you know our baby
and we'll just you know keep it here and
nobody else can have it they're just
like let's let's share that with the
world and let's you know make it all
better because it's we're all it's so
early days and all this stuff
is a lot to figure out and I don't know
who the winner is going to be it maybe
we'll all be winners maybe none will be
but we want a you know I'll just work
together and try and be healthy and get
there all right
okay so we'll open up to questions if
anyone has any all right I'll just hop
down here with this microphone hint to
you so you guys are on the front lines
and you'll have a really good
perspective on this I'm curious what you
which projects you guys think belong on
chain kind of where you draw the line
for that and you know as assuming that
we improve efficiency and scalability
and stuff do you guys think that will
change so how ya talk first yes I have I
could flick to there sorry I I like on
changes because it's sort of this
singular place to integrate with you
know the title is integrating with
etherium and as we've talked about it's
already sufficiently complicated and the
more things we add on top of that
especially layer to things and various
protocols it becomes even more
complicated so the sort of onboarding
cost gets a lot more expensive and it's
not as easy unless we build tools that
can somehow do this but if you know now
some new protocol comes out that you
know all of the exchanges all the
wallets need to integrate and you know
you scale that up you know it's a
multiple different ones that's going to
be fairly difficult and so keeping it on
chain you know has some nice guarantees
there just simplifies it overall but
clearly that has sort of you know
scaling in limits and stuff and there's
a lot that can be done off chaining and
maybe you know published there as just
some proof and I think there's also a
lot of value in that if we can have you
know really good sort of second layer
solutions to to to pull it off yeah so I
would give it like I think there's a
trick answer to that because you sit on
chain I think like side chain right now
it's the best because you know with side
chains that checkpoint it into the main
main that you kind of get a lot of
advantages that of being on chaining
that it's really easy to integrate
because people can see you through
transactions and you've got an EVM on
the side chain so you can run quite a
bit there but it's not going to scale
eventually still I mean a single that
could still take up you know all the gas
on its own side chain again so
but for now I think where we are at
that's that's that's the best you have
and it's gonna get us there 22.0 yeah I
kind of see both sides of it like for
simplicity doing it all on just the main
aetherium chain it's definitely the
easiest way to keep it simple and
scalability purposes it's definitely
nicer to do it on a society of some kind
there's I think some kind of balance
there I would at the very least I think
anything that should be anything that's
considerably expensive I mean you can
like crypto katie's anything that clogs
the network or is sufficiently expensive
or has sufficient network should be in
some side chain or shard or something
off the main chain of course anything
more simplistic that maybe either
doesn't have traffic or like I'm not
sure if it makes sense for like new
developers coming in after like 2.0 jobs
and stuff like that should jump right
into oh yeah we're gonna develop our
thing right into a sidechain this is my
first my first app is on plasma and it
it increases the barrier to entry for
aetherium which is one of the things
that makes aetherium so great in my
opinion so there's a balance there
depending on in each individual app
alright so I think you your moderator
asked a great question earlier about
what was your favorite feature of a
theorem but I'd like to ask the opposite
what is the the biggest pain point that
you as scaling if your room developers
deal with I know that Morgan from
shapeshift you mentioned earlier that
dealing with the lack of return codes
was a pain point is that the biggest
pain point what's the worst part about
dealing with a theorem at scale oh so I
think there's really two sides of that
is there's the solidity side in the
smart contract and what's a pain point
of dealing with those and that that
whole ecosystem and then the server-side
architectures and integrating at
addresses or do we choose a wall in
contract personally having doing both
sides yeah era codes is a huge pain
point for me it's it's a huge time sink
trying to debug issues there
when you have essentially two maybe
three error codes that all basically say
the same thing it's definitely a pain
point I think it's the internal
transactions challenge where other
people are not viewing them easily for
me because that you don't have much
control of yourself you can yeah no
matter how much time you spend yourself
you know gonna get someone else to
accept a transaction or is diecuts yeah
I think there's a few I think like
that's a huge one that's like prevents
people from sort of getting on board and
getting there I think there's like
there's things we can do that will make
that make that easier and these are
things that like eventually once you get
them kind of working correctly well they
do become not so much of a pain point
but they're hard to get right the first
time
things that we see like you know
regularly well I'm gonna say two things
one thing we see regularly not much
recently it's everything's kind of died
down but all the gas price stuff like
really this year all the time gas prices
spiking things get stuck and and that's
a pretty big pain point and we don't
currently have systems on aetherium that
sort of automatically recovers that and
we talked a lot of some ideas earlier
that could help with that um the other
one that's more of a rarity but also
similarly for like building these sort
of high-volume systems if you have to
start tracking these nonces and stuff
yourself if you get one out of order
you're in a really weird spot because if
you're doing like multiple sends for
customers there and you like skip one
now you have so if you have like four
five six and you don't do five for some
reason it'll hold you've got this whole
and so the other ones just gonna be
pending forever until you fill that in
and like what if you like I don't know
we tried something else and then you
send another transaction out you can't
cancel this six now like it's out there
and so you have this weird like like we
can't introduce like canceling
transactions that would we eat like not
really possible but that sort of thing
in like calculating these nonces like
you can't because you're not just using
a single node you have to sort of track
that yourself and make the right
decision there and so that's a challenge
to like do right and then when you get
it wrong specifically going too high
going too low the node will reject
usually it's just like you know it's
sink that but going too high creates
this thing where it's like oh yeah now
what and you don't know how long it's
gonna last it's in the nymphal the other
thing once where you know we filled in
the gap in it went out we didn't expect
it to and it's like oh well
it was okay but not that a fun thing all
right any other questions out here all
right we got one way at the back gonna
run I know
so a better standardization for like
your server architecture or or smart
contracts themselves air close where do
you guys feel the governance process is
for that are you guys talking one-to-one
to each other or is that on the IPS or
it's kind of like how do you have those
conversations together in the industry
to get stuff done
yeah I'll see your question one thing
real quick on that like this that's one
of the reasons why we wanted to put
together this panel is to get people
talking and people talking in public I
think there's a lot of sort of secrecy a
lot of things are shrouded and they
shouldn't be and there's no real reason
for them to be but I think we all
approach things different ways like if
we were to talk about how we all run our
nose or how we all did not stuff I
imagine we'd all sort of do it
differently and there's things to learn
from each other there like we've already
learned things talking about you know
the contracts like sprint trade-offs
there and the way they do things in the
way we do things and I think the big
thing is yeah at least get people
talking I don't necessarily have in a
year I think the best place to do that
there's definitely you know something
you know just every now and then you
talk with somebody or every child but I
did think it'd be really cool to see
more sort of open spaces to discuss this
things especially with sort of you know
like big participants of the network
being able to contribute to those
conversations maybe I think it would
depend on the company because I think
also companies like to do a lot of sort
of protecting of their information too
and that becomes sort of challenging but
it's really it's like all open
conversation and it is kind of segmented
that conversation right now whether it's
get or stack exchange or conferences or
github issues there's tons of places
where a lot of the same conversations
are happening so I don't really know of
any centralized place where this is all
happening as of right now but it is all
about really conversation and seeing
what other people are doing this is a
very new space
we're all kind of just figuring it out
and a lot of things you almost have to
learn from either experience or
listening to somebody that did get the
experience and make those mistakes
already do you think think interesting
because it is the thing like I don't
think how people talk about enough is
sort of the the types of problems we
were talking about today I think a lot
of the people working on their protocols
and stuff tend to be very low-level and
close to there and have you know
specific ideas around how people are
gonna run notes and stuff together but I
think that you know at sufficiently
large volumes you have to do it
differently and I don't know of a space
where people really talk about that yet
so I think definitely each research
would be an interesting place to just
sort of discuss that or either magicians
or any of the sort of forms like that
all right we got we got one quick
comment here just another vote in favor
of just some sort of central group group
chat I don't care at the venue but when
I'm having issues with like all these
like holes and nonces and sending
transactions and like all that stuff
just if we can find some central group
chat ever I can ask you guys actually
are a few at the same time when they get
too large and people start leaving and
start tuning out so it's it's a
challenge yeah I think it would be good
for us to blog post or something about
this so we've only got 30 seconds I know
the panel's out here but to end on an
interesting note it sounds like there's
a lot of challenges from a developer
point of view I know that's what you
engineers love but what like why are you
here what's so exciting about you to be
in the space in five seconds
none of the challenges are
insurmountable and we're seeing the 1/2
scale and the one to get better here and
I I think the community is great yeah
I'm sort of the privacy and sort of
ownership of sort of your own funds and
you know all the sort of developments
there like fascinating to me all this
sort of talks today about all the
privacy all the cryptography and stuff
is it's interesting I think can just
lead to a lot of you know good in the
world in people sort of you know all
right we gonna we gotta kind of wrap it
up here Morgan if you want a five-second
quick brand new credit set of problems
to solve that haven't been solved
before and we can solve them together
you