[Music]
again hi my name is Demetria I'm working
with a foundation for about three years
and for a long time a serum testing was
performed by one person Christophe judge
he neutrally invited me to the Assyrian
foundation and then I continued his work
so today I will describe you for you how
the testing process goes in the serum
and what we achieved for the last year
science Shanghai so what we have is a
yellow paper which is a protocol yellow
paper describes the serum protocol what
it doesn't make it easier to understand
actually this picture represents yellow
paper really well nobody able to
understand this unless we tell it my and
we have lots of implementation of this
protocol written on different languages
different clients and the testing
problem is to make sure that every of
the client is in consensus with each
other and they interpret they
implemented this yellow paper rules in a
way that there are no consensus issues
between themselves how to do that we
have our testing sealed each test file
is a JSON instructions encoded in JSON
file and we could select three main
times of the test so the first one is
the VM test is basically number of
instruction for IBM white machine it's
the most simple test in the whole test
year so if if there is any developers
here you could
if you implement an EVM you could start
with VM tests a most simple ones
the next either state tests is a bit
more complex and VM tests it has
accounts transactions and state changes
state hashes and so it's a bit more
complex and the most sophisticated s is
a blockchain it has actual mining
difficulty and those algorithms
implemented well and for a long time we
require to all the clients to implement
custom tests though
suppose they had to read this JSON files
and implement this and compare the
results it was not so so good so we use
a C++ client to run a test source which
is my job is to keep those sources for
test sources updated with the most
recent issues discovered and what we are
trying to achieve is to create lots of
lots of different test source files
describing all of the possible situation
which we could get in a VM code
execution and then C++ clang execute
those instructions and it creates a
final test final test is being processed
by other clients and say compares their
results to the C++ clients and that's
how we see if everyone agrees with C++
client which is like a golden star user
is an issue we open up these put and
discuss whether we have to make a change
in the C++ client or other clients so we
had a couple of problems the first
problem is to create an ultimate test
which covers all of the code or all the
possible situations which could run and
possibly find all of the bugs and
concerns to see here's the second
problem is that clients they should
execute our tests and because before we
are required to my exact clients this
custom tool
I had to manually ask every client
developer have you run this test have
you updated the test repository is it
updated and one time I discovered that
core client wasn't running the recent
tests for about half of one a year and
the ghosa there was a heart for cover
radiantly it still not cut medication
abuse the test not to mention that I
don't know the status of other clients
[Music]
so how we solve those issues to get the
best code coverage just we have tests
cases list which could be found on
Google Documents this is the many o test
cases if you came up with an idea for
test case you could manually add this to
this also I want to mention if you think
you know how to make a consensus Asian's
is a reward for bug bounty from the
foundation if you discover a consensus
this year you could get money and so to
cover even more cases you can you can
possibly come up with every situation
but I think it's impossible so we did
some tools that basically run a random
code random even bytecode on not really
random like fast testing is a smart
random code and this code hope we hope
that it will find some consensus ratio
and it did another approach will be to
use a fancy tool that you analyze a
source of a client and then run
different white-coat executions and
maybe some fancy other reasons and
you'll produce a very very good test ok
two solves the next issue is the clients
being updated last year i mentioned that
we have a hive tool in development and
now it's life and working now we no
longer require
time to implement a custom test oh they
just need to provide a client and an RPC
interface and then any client could be
integrated in a hive tool and this have
to perform a test execution
automatically and we have results equals
C on a high stats page there is also of
all the rest execution on every client
also clients have continuous integration
bills in a github each commit executes
this continuous integration test in a
github which synced with our test wrap
or a serum tests and I still personally
do ask people on a Skype our channels
versus a pay phone most recent test
executions another field of work which
was performed by Casey he did
standardization of IBM logs and by doing
this we could now compare that not only
we have the same hash after the test
execution but the way we get to that
hash is exactly the same on every client
and for the past year we added some
debugging for the test
if to which I'm a developer and it's
basically a whole new field of work with
those traces for again and better
debugging okay and another update for
the way for this year will be my
personal for coverage by fork I mean
serum has certain hard work points like
constant and other proposals which we
actually don't have a consensus about
naming those Forks I called like EAP 158
and others codes and some kinds of beers
dragons and those points means that from
this particular block the protocol has
changed and we have
test that all the changes and all of the
clients implemented correctly but we
have a test cases and there I had many
only to copy all of the test cases for
each heart Fork
and then I came up with an idea of
general test format so now I described
one single test case and then C++
convert this test and executed on every
fork and produces a couple of output
final tests it's actually a result of
this particular test cases on every fork
that that is what I call general tests
and for the past year I realized state
test which is now general every status
being executed on every fork and in
future all of the test cases that we
have will be executed I just have to run
one comment and all the test cases will
be generated on the recent Forks same
with blockchain tests and next will be
the transaction tests also thanks to
your G he did awesome documentation
about how he learned because he has
joined us this year he created the
documentation of his steps when he was
learning the process of test generator
generating the test and then he graded
this documentation if you interested in
running the test execution yourself or
if you are developer and you need to
know how this works
there is no updated documentation you
could take the pictures of the links and
you could see that now our testing team
is not a single person we have like lots
of people joining and most of them
joining us this year
so special thanks to all of them working
together on tests okay we still have
time for questions
oh yeah not all of the tests are running
on a - portion they're so general state
tests are being converted in a
blockchain test this is another update
so I just run a common and although
general state has been converted
blockchain tests which then will be run
on hive okay thanks Dimitri
[Music]