[Applause]
[Music]
this is Harry's hello we work on
truffles and I wasn't about your guy
we decide you gonna get through then
we're gonna talk about the current state
of smart contractor baggage I just gonna
talk about debugging data formats what
are they what do they do
why does every other computing platform
use them and we're going to talk about
some of the design challenges with
coming from a solidity perspective but
for smart contracts in general then
we're gonna make this interactive so it
is a hopefully we'll break into groups
and kind of talk about what would it one
of these debugging data formats look
like we'll see if people are not too
afraid to share their findings then
we'll talk about specific areas of work
like what one of the specific challenges
is not from a high level but
specifically break into groups again and
now settings again no one ever want to
go
it's trying and then we'll complete so
you see how this all goes did that
already hurt they lost ischemic our code
[Music]
now thumbs down
great let's do bugging like on aetherium
today well we have solidity source
mappings these are extremely useful they
are fantastic you have some bike coats
literally gives us bytecode by codes we
can model it as a list of instructions
and it's literally gives us these source
mappings which correspond to a
particular byte code and we think of it
like an array there's an array of
instructions and there's an array of
source maps and for each of the entries
in the source map it corresponds to the
one byte code you know the corresponding
byte code instruction and it tells you
the file the end the source range and
some other stuff I don't fit into that
then what you can do is you can take
this source map you can take the byte
code and you have some running program
that you're looking at you can get the
trace you know you look at the execution
of your smart contract transaction you
see everything that happened on the
opcodes and you get this company called
a program counter the program counter is
where where am i and the currently
executing contents what you do is you
take the program counter you convert you
figure out which bytecode instruction
that refers to so if you have you know
you're a program counter ten maybe
that's wherever it is in the bytecode
you find out what instruction number
that is then you take that index I or
whatever you go to the source map that
tells you what you need to know so
that's pretty handy truffles are always
version of our debugger that's all it
did is it translated from the low-level
evm opcode instructions up to the
high-level source
this is what all of these markets do
these are the three dividers that I am
aware of or solidity I don't think there
are any other high-level Imperium
languages that have debuggers today
apologies if I'm missing something
that's hard work to build a debugger so
I hate to leave anyone out but we have
trouble debugger capitalist not my first
remix has a debugger and the meadowsweet
has the debugger so these do well they
figure out how solidity works through
trial and error and yeah well so
literally does document quite a lot like
storage allocation program does not
figure things out the fraud letter do
you built the program figure things out
right yeah so we can't decide travels
debugger how do you figure out what a
variable is well you observe it in the
wild and the reason for that is because
solidity is no way of telling us so
hopefully we'll change that we're down
great so of course it's not like
solidity gives us nothing because that
mean truly would be pretty stuck
hopefully it does give us the abstracts
and texture
and so you know for those who are not
familiar right you parse the code into
it into a structured tree which contains
all the loops to contain all the
information and it's you know much more
received machine readable workable
rather than a giant string and so we use
this for a number of things and in
particularly the music for locating the
variables um we use it to determine
where the state variables are located
and we use it and we combined it with
source map right so the source map tells
us where we are in the source code
considering the source code has a string
but then we combine this with our
knowledge the abstract syntax tree that
the debugger sorry that's solidity
compiler provides us with which gives
these source ranges on each no and we
use this to determine not only where we
are in the source in terms of a string
in disease but also in terms of what no
rod and so using this we can do a bunch
of processing to determine where the
local variables on the stack are and
also other things but note there
portrait was relying on the abstract
syntax tree we're also lying on our
particular knowledge of how solidity
stores data which is not supplied to us
in any format that's just stuff we
programmed into the debugger yeah yeah
figuring that out slowly yeah how's that
sound good build a bugger
it doesn't work on other platforms the
pile C program how can you do bug that
well there's this concept called a
debugging data format which comes from
the compiler the compiler provides the
information about a particular program
and says well it gives you that
translation right you can given any
machine State here's how you can get the
high-level source code state so it's
like give wonder bites you stack your
memory of storage here's how you can get
variables out of that and this is
extremely useful there's a number of
these there's dwarf is probably the most
popular you can read about it it has a 5
477 page specification it's a binary
format
yes so you have these concentrate so
here's a simple program that these are
related examples that defenders took
them from that paper but you have these
tags in those sodas subprogram tag which
says oh this is this is food and there's
a variable there and the very long is a
type and it has a location and health
and even your clear stuff so you can say
we're going to be dealing with this so
the compiler will say this is an INT in
size four I guess it's all inside right
so you do that you say oh what's CC has
is a base type of an int as a particular
location this is the frame pointer
offset so it's when you find it it's 12
items from the top of the stack this is
pretty useful right but you know this
gets encoded as binary and gets embedded
into the output right if you've used a
traditional computing language and you
output it the debug information and
using GCC for instances like - but you
you specify a flag that says I want to
divide the ampulla and you get a binary
that's twice the size or three times the
size and has all this extra information
which you know you would then if you're
in you deploy that and you paid three
times too much and you've seen the block
gas limit you know it's all set to parse
this
but I mean work works pretty well
there's there's others too I mentioned
them here there's PDV as Microsoft's to
use it as a proprietary format it seems
to be better it lives externally so you
get a dot peb file rather than loading
your actual binary output coffee is
interesting because coffee is of an
actual binary a performant that's what
the compiler gives you regardless but it
includes the specification that like has
de buggy information so you've got this
binary in the binary may or may not have
the debug information whereas the work
is like Bolton on that was the separate
standard if anyone's familiar with e.l.f
binaries that's the linux native binary
format that's why they got the name for
alright so here are some requirements
that you come up with naturally it has
to live outside the bytecode give gas it
should be easy to generate because I
want to make this litany of developers
lives terrible and it must be easy to
read because I don't want Aries life to
be terrible and it should be extensible
because we don't only want to support
solidity we also want to support Viper
and all the other smart contract
languages and also you know he was
coming and that's gonna break our
debugger
so get your super fireman suite there
probably others anyway
Mary tell us why it's difficult okay so
there's a lot of guns that going to uh
designing for these and actually post
these for questions probably the one
that make put last year
what's this format even look like now we
take inspiration from dwarf now Nick
guys make sure you dwarf uses this idea
of declaring types so yeah so like you
can declare types and then add variables
we of those types and that's really nice
because it's like you know you're not
limited to this preset you know just
like a few preset types that would be
not very useful
maybe there's sort of this the
alternative approach you know if you can
here prepare types and you also don't
want to limit yourself to and depending
if you're kind of stuck when it comes to
like and when arrays or sorry I didn't
explain that well you can start if you
can't declare types and you have to
associate a separate type with each
variable then you get kind of stuck when
you have to handle compound things like
a race like are you going to you know
associate a thing to every entry of a
dynamically sized or alien no you can't
do that so this ability to their type is
quite useful and we will like to want
that oil I don't know I don't know much
about designing data but data format but
there's all sorts of other challenges
and a lot of these by the way a lot of
these points are fair our are inspired
by solidity at least you know you want
to be more general than
these are particular talents that come
up with solidity in particular and
therefore and if they come up with a
person like if we need to solve these
just handle solidity then we're
certainly gonna need to solve these Tom
so location dependent representation
salinity represents data differently
depending on whether it's in storage in
memory in call data or on the stack do
we you know to meet in the up just make
it so that to where when you declare
tight you have to describe its
representation in all locations or maybe
between these it's four different you
know it's three or four different types
you know declare mole separately and how
do we represent the way that solidity
does dynamic arrays and mappings and
storage which has this uh you know catch
act based location scheme that's like
it's like you guys wanna be putting that
in your debug data format like like I
mean yeah we can put a flag for it or
something but it's like it's a little
it's a little bunch maybe um Nick here
has this whole idea that we should make
button data format maybe maybe less data
based and more code based and you'd have
these common extension this baby would
be a good candidate for that like a
mapping accessibility mapping extension
where the solidity compiler would say
this is a mapping into int or into
address or whatever and whatever
debugger builds its own implementation
of what that mapping means and you know
the the solidity compiler would make me
provide you know the natural language
represent like a natural language
specification but when the translating
code and then it would be a primitive
address book perhaps to be a primitive
but you know wherever there's a gap how
do you represent what a mapping is
primitively well i mean permanent is the
wrong word here
but well so maybe we should who here has
seen their what depe way that solidity
does mappings under the hood I don't
know if we should get into it yeah so
I'll get into it right now I had to get
through all right yes so then you get to
relatively what about call data pointers
pointers and call data are relative but
they're not relative to themselves their
relative to the beginning of the
structure that contains them so that
potentially poses some problems out of
it represent composite types like how do
we need to give you presumably we need
to give the people using this more map a
way to specify in composite types than
like how they work and these could be
structs but they could also be things
that in solidity are our primitives but
that are logically speaking composite
such as for instance an extra function
pointer which after all consists of an
address and a selector or an internal
function pointer which table consists of
two separate values a program counter
plug for the constructor bytecode one
for the deployment it's really really
interesting how its internal function
pointers are in solidity like how he's
saying is that it's an 8 byte value
where the first byte is if for like
first 4 bytes are for the constructor so
you know you have a pointer to some
function and like a you know an internal
function you want to store that in a
variable it's like maybe you have a math
function that takes another function
if you need to point to that in the
actual evm bytecode you just you have to
point to some some byte offset but with
smart contracts you have the constructor
byte code the code that runs when you
deploy the contract and you have the
runtime byte code which runs when you
send a transaction to the contract and
they're not the same byte code so if you
want to point to a function you know you
have these two separate byte codes you
want to point to some words you need to
represent both of those anyway moving on
well this second last one I've already
talked about
where's how do we draw the line between
you know to what extent to be water
reliant home base extensions persons to
what extent do we want to rely on data
that you know and just building that
stuff into the format I have had
differing opinions on this but let's
save that for another time
and the first one how do we how do we
determine how many members that we're
local variables are not a fixed location
right I mean newest state variables
these live in a fixed location local
variables these live on the stack it's
going to depend on you know what's below
you on the stack how far up monthly
weird dysfunction snap frames start now
you know if you're used to running you
know things on you know x86 or whatever
it's like oh you look at the frame water
we don't there's no silly personal
three-pointers in the solidity there
there are no free pointers in solidity
other words maybe to try and write a
second explanation let's say you call
function when you call that function it
goes on the call stack as a frame and
there's a bunch of variables maybe your
function declares a B and C those three
variables go on the stack and then you
have some working space for like maybe
you do some complex expression and you
have to add and multiply
that's Armand so you have a bunch of
other bad loops on the stack on top of
the variables you have to know where
that frame begins
why can you get frame pointer I'm some
nosh-nosh there's no it doesn't make
access to the stack
oh that's a really good point I mean
they're thinking about adding those
opcodes but they told ya those in those
simply do not exist in like there's no
way you could've anticipated that or
like you're like oh yeah we're gonna
wait for them to add dynamic access to
the stack like yeah that's actually a
really good reason that I didn't think
of oh well okay now we're not dealing
the freeing point your son yeah there's
no point also I want to make the tag
onto that is that salinities of a
modifier feature poses additional
problems which I'm not going to go into
right now because that would take us
well our fields suffice it to say that
they they complicate the matter great so
these are some of the problems we want
to solve anyone else have anything else
I can edit the slides down and anyone
know yeah maybe also take an approach
from the other side they've let's start
how do we do it but rather what is the
information we want to have during the
debugging process oh yeah absolutely
anything else
and that another thing
oh I see
there we go Amy
right I'm sorry Chris gone how to deal
with in the mediator presentation so if
you can fight through you'll do we also
want to represent excellent
so if you have a valley on the stack we
want to do only video we only want to
know which certificate variable that
represents or we also want to know which
you will but ever since
[Music]
that's interesting them to do that the
same in the same location like oh this
is this variable is this variable we
want it to be more general than just for
solidity and part of the idea of having
a debugging the apartment is to help
provide that decoupling right because
right now our debugger is specifically
at the solidity to Dugger it relies
heavily on specific knowledge about
slitted we're hoping that with a
debugging data format the whatever
compiler could provide a lot of
disinformation in the format I mean as I
mentioned you know I talked a lot in the
other slide about solidity specific
challenges but you know the idea is that
any challenge we have to solve for
solidity is at the very least a they
probably have to salt lick the problem
is at least this hard right I'm not
trying to exclude other things I'm just
trying to list like the problem is at
least this hard yeah to your point you
can look at this you can declare these
base types and this F work works work
works on any language you you need a
pork output on the scene import help it
on Pascal and so on and it kind of Rises
these mechanisms where you can specify
that an int is sized for four bytes but
you know on another platform or on
another language you might have eight
byte integers so the dwarf provides
mechanisms for doing this but it doesn't
get as complicated as say salinity
storage arrays versus solidity memory
array is like structs insulin via memory
which one ones tightly packed ones not
coming back to store integers Oh
storage is tightly packed it's the
storage arrays are typed
the memory erase are not tightly packed
and so how do you represent that with
something like this perhaps it is
unclear I mean that's not even not maybe
even cook say that I'm not sure I'd
here's an it here's a question for you
is do you represent that is a property
of her upper race I probably would have
represented it as a property of the
underlying types like I probably would
have said oh yeah and in memory if Dade
is 32 bytes even though it's called an
end date and in storage means B is 8
bytes but you know both those are viable
representations so you need to figure
out which one to go with because there's
other differences between memory erase
and storage array it's like I mean I'd
say the bigger difference is how they
handle reference types but neat stuff
all right so there's some links for you
how you wrote a paper that documents all
of salinities data representation yeah I
mean those of you who work on solidity
I'm just recapitulating all the work you
did and putting it in a form that I hope
is readable but up as tables here my
tables they are so truffle is about to
release a library called truffle Kodak
which handles any coding and decoding of
values not only solidity values but also
API values and you can see and we'll
just kind of show you what dealing with
we have this representation of all
values in solidity this is the output
format that Trump Lakota will be
providing and these are JavaScript
objects and you can see we model the
types so each result will have a type
field in a value field and the type
field represents one of these you know
generally tight yeah I should know that
this is very that this Park
thing is very salinity specific but so
so don't look to this too much
necessarily for the debugging data part
because that we really do want to be
more tenable right but this is a start
right
because you can say we have a struct
type well let's just do like a dynamic
array right dynamic array has a base
type which is another type so that might
be an integer and it has a location
which might be a memory storage called
it out of the staff that's a raised
again no constant but yeah so those are
types and then we also have values which
represent the corresponding value so you
know a pair of tightened value gives you
a representation of your source variable
or whatever they trying to decode so
maybe the at least something our hope is
that this kind of approach will lend
itself to a more general solution
currently this is not serializable
because data values this limine can be
circular and we can't really serialize
them to json very easily so the more
work you done for Shore
and then of course shout out to these
soliciting dogs for being excellent
oh wait am i all right anyway that's not
that's not relevant circuit circular
stuff is not particularly relevant
particularly relevant right now we have
all right so I hope yours we could break
into groups and kind of talk about a
high-level structure like what might
this look like give us 20 minutes for
that unless everyone is bored you know
come here with dwarf maybe we can dig
listen I can't a sign at least maybe we
can get the spec under 400 pages so it
into ARF the the top-level concept they
call the diee of debugging information
entry so that's the thing with the tag
each diee has a tag and then the tag
each tag has a number of attributes and
so I don't know maybe a start here will
be to figure out how we can do this in
JSON I'm gonna scratch paper
I don't know I'm talking about your
cells there's a lot of people here yeah
I was expecting tables but now you just
have chairs all right I'll give everyone
some people are talking design a data
format today
it's perfect oh yeah sure alright maybe
just maybe it's what's the top level
[Applause]