[Applause]
so today I'm here to talk about
validating verticals and then a neat one
and when they need to level using AI
agents so we focused on the ant no
matter what I'm learning and you may be
wondering why are we looking at this how
does it fit
I am Shila me and I'm gonna explain all
of us and talk you through that so in
the past we've looked at something
called selfish money which is a strategy
attack on IBM through one boy next
vertical where essentially you're trying
to attack the network that employees the
proposal calculation and you do this by
essentially keeping like a chain that
it's a head so essentially you don't
send your blocks as soon as you receive
them but you will hold them and
sometimes repeatedly as you keep it
heads from other - and so the end you're
gonna make the oyster time by looking at
a loss that you just sent and you're
always keep ahead so they do ensure that
you're winning and so this is a problem
that's they've studied and different
papers and there's different algorithms
that you can follow as I described
before you can hold a number of Bluffs
and whenever you how elite if someone
sends you a block then increasing the
general heights you publish a walk that
and essentially always keeping a heads
up in to ensure that you can maximize
aware that you getting because when you
something you either get a maintained
reward or an awful reward which are
almost essentially the same and so I
learned with the Pegasus team at
consensus and we focus on research so
make develop a framework to build
simulations for different types of
particles so we look them through from
work for the state and so we decided it
would be pretty interesting to move
implements a selfish attack and look at
the data that wouldn't return so this is
a bit like oh there looks like we
actually took the combination of the
difficulty we're looking at how it's
done if you're really just a kind of a
sense of what their reward time and so
the only what you're doing as I
mentioned before you can get a reward
from mining a block which is right now
to eat when the transaction is included
and if you include any animals that you
receive so your ordinary - 175
125 or - and since the transaction fees
that you receive this quite small
sometimes we decide to not include this
in our own simulation and so we decided
to build a test case where you only have
a network of nine miners were always fun
and
liner that is selfishly mining and so
what happens if that there's some
variation and we decided not to internal
warrants of transactions and some of
their results on murder this so
essentially what we found is that once
you have to like this time of the
hashing power in the network you can
start attacking it and doing selfish
mining and this is more beneficial for
you because this manipulates the
difficulty calculation and it increases
your own work so they're more hashing
power you have the better for you
and so if you wanted to test how would
this prove if you are giving it to a
reinforced not an agent and since we
have actually active back it up we can
compare the results and see what we
would get with it and so that's why you
might be wondering why are we doing
reinforcement learning when we were
focused on watching but if this will
give you like a sense of what
reinforcement learning is in case you
haven't heard it is the concept where
you have a piece of software it's an a
gendered construct for an environment
one of the most famous ones is deep mind
where they implement in a deep Q network
and essentially interacting with
different attendings and it learns how
to solve it so we thought okay we can
take this concept and implemented into
different protocols and just see what
happens
and then just like make a sense of why
we chose that too is because it's an
automated system that essentially has
proven to beat humans and many other
elements and strategies so it just made
sense to go that route so how does that
work you know this room or you have an
agent that is collecting ice box
the environment so it's partial
information on the current state and
then you have actions that you can take
which will return a reward based on
those actions and so the process into
using some loop where you just keep it's
raining two different actions in the ten
states and so what are the different
observations that you can get in terms
of our product over looking at the
number of blocks mine the number of luck
receive or something all those different
things that you're actually doing in the
network and feeding that information
into the agent so they can learn the
impact of their actions
and so it's really still blessing but
they don't want to be too broad in terms
of wider minor can do so we decided that
you can only combine up to ten blocks
hidden in your own private chain and you
should be able to publish up to three
blocks in a row so you could publish one
two or three based on whatever it's
rented al seems the best and then you
would get some reward based on that and
so I think I'm also making a part of
setting this up and strength understand
what is the best promoter can you go to
an agent because you wanna give her more
such that it understands what's the
optimal strategy to think it also cause
it was golly selfish mining without
necessarily implementing the entire
Center
Institute and so what we decided to do
in our case is to incentivize staying
ahead just minding as many boss as
possible faster than the other miners
but there's other stretches that you can
take we try to test in a few to see what
is the most optimal result but
eventually justice intervention using
the metric of staying ahead that's
probably the best and so going back see
what this looks like we have on this
agent that has been implemented in
Python and his colleague code that is
implemented in Java this mountain
simulator and it's essentially going
through this loop over and over again so
I think actions and then collected the
information that he gets from the
simulation and so there's a number of
different alternates that you can use as
I mentioned before like one of the most
famous is the network's many fiddles 103
Q learning algorithms policy 3 based
algorithms they all play on different
sets of statistics and modeling and use
of light chains as the back of how they
calculate different states and wards and
so I'm excited that probably the
simplest one to go for was Q learning
and so illustrate and explain a little
bit what Q learning is essentially so we
use this Python framework that's
developed by an opening eye and it
allows you to create your own
environment and interface that you can
use and so we developed that calling the
Java code and then we tested for men
so we wanted to see what was the best
strategy when you have 10% of the hash
around 40% and 60% and then we set some
hydro parameters that are related to
these Q tables and how you calculate the
reward so you look at the learning rate
which the session is just telling you
how much you love you and new
information that you receive versus
information that you've recorded in the
past and we also have a discount factor
which also pertains to the reward you
want to look at the current reward that
you have for an action plus there were
work for making it a next option because
you're trying to discover what is the
most optimal path and so it's something
that you have to - and in terms of how
do you value the future of the Lord
because you can think okay I kept mine a
lot right now just have it as soon as
possible
and again or if - but if I decide to
wait a couple of what's ahead
publish it there's traffic was buying
some money might cost less because of
inflation or you can just think well I
lined the four blocks in one minute
that's a negligible it's worth the same
so based on what you value most you will
tune on this parameter and then finally
you have epsilon which is just related
to randomness how many choices do you do
that our random person's time a choice
if you do that you're maximizing your
strategy by thinking that they
and so there's two states as I mentioned
before the first one is driven by
Epsilon
where you're just trying to make random
choices so the different actions we can
take and reporting the results that you
get in this table and then this will
allow to generate different valleys so
that you can reach multiple in deciding
what is the best action at each state
that you reach and so then there's a
theory during this that you saw some
people development equations and it just
said that basically by maximizing at
each state that you find yourself what
is the optimal action you will maximize
for the entirety of your decision
process and so you're just looking at
maybe two before their reward and then
you're also adding the value above the
future award of the next state in action
but with an aesthetic factor so that's
what I was talking about before that
zero point about you and so does the
reason is filling with more concrete
because it can be very abstract we can
think about this Q matrix being
something like this where you have four
different actions so you can hold your
block and not publish it just keep it in
the branching you can polish one block
two or three as I mentioned before this
is the difference
some actions you can take and then you
can hold one two three four and more but
for simplicity in this example and
that's it three and so when you start to
initialize it and all the values aren't
zero and as you keep without going
around them the connections you'll
change the values so let's just walk
through one of these examples so let's
say that you start and you start in that
chain you decide to hold and so you're
gonna calculate what happens when you
hold and then maybe the next step you
want to publish it so that will generate
a reward let's say to me and so you
continue doing those few random paths
until you generated enough values this
is like random values not the actual
ones but just to give you any cable what
happens and then after a certain point
what you start doing is that okay I find
myself that I speak about holy - so I
have two blocks wise my next next action
sure polish them or should I call them
and so you think the maximum value for
each action if that's how you can find a
possible path and so I don't mention
briefly there's a the marker change
crosses bet that's all this information
and ensures of this equation holds true
and it's just an ATS so that you can
also visualize this in
wait you can have a chain where you're
mining and then you can serve you won't
write a chain and keep switching between
one and the other and that essentially
because of the properties of our chain
use you reach the steady state and
that's why this come table is useful
because you can assume that to exert a
number of steps now you've reached the
same and you were off to infinity and so
that's how you can maximize the values
and so there's a lot of attention
challenges we can get this approach
because you need to understand what's
the best data that we eat you need to
test for all these hyperparameters and
keep changing them until you can find
something optimal and it's also a
consideration of how do you build this
what are the behaviors that you
incentivize how do you to find something
that's visiting and so on there's a lot
of things that you can buy challenges
with and so it just to go back to where
you can look through some work we took a
survey shion's we were distracted what
is the number of blocks that you've
published in the public chain private
chain I'm keeping track of all calls and
forts and then letting it know so we can
make the best actions and so we ran this
similar to what we were in in the
self-contained simulation lying on its
miners and their one is
my nerdiness top date through the
reinforcement learning agent so we
measured it at 10 40 X 60 percent
hashing power and we collected the
results and so as I mentioned before we
use the simulator and John we use the
spray wearing company I - so that's a
big publishing memory libraries and then
I ran all of these tests in a double us
so that we could test it collect all the
data and so this is just an overview of
what we were doing and the different
hashing rates we collected the reward
only a total of one hour remaining we
also collected over the total of
episodes or simulation so essentially
all raw element my agent run for an hour
or until it has mine a thousand watts
and record that behavior and I run it
for ten thousand episodes while you're
changing the actions that you select and
we also measure for three different
strategies we just wanted to make sure
that we weren't performing better than
if you were just mine honestly or if
that your mind just randomly and we
wanted to prepare the results so whether
there was also play we found it when you
ran for even just like a thousand
episodes the agent that was incentivized
to hold his chain a ten percent
for plumbing work because they're all
ideas that I've done percent you're not
getting are gonna be ahead of all the
other miners who don't have enough power
so you're just in the losing one however
for any person they already start
gaining an increase in your reward after
maybe like five hundred sites you can
already see a little bit of an
improvement in terms of what you're
doing and then at sixty percent it's
just like taking over the network so
their results I never had that here
anywhere else for looking at this
nitrate which is the ratio of lost of
mine so this is how many blots that you
mind versus the number total watts mine
so what you see here is that for ten
percent you're doing worse than if you
were just being honest but forty percent
though you you started moving we set two
different reward systems just testing
and for one of those starting to be
better at twenty percent while acting
selfishly so he's getting like almost
50% of the watts which is gonna slay
increase and at sixty percent is just
mining all of the boss because you get
such a huge advantage from selfish money
that essentially you're publishing most
of the bots because nobody can catch up
to you and what happens is that you're
also decreasing the difficulty of lot so
you can my investor with the same I
should rate and so what are some of the
key takeaways of all of this is that at
sixty percent you're doing a lot of
damage but we all know that even like I
thought this interview should be able to
move a lot of damage a forty percent you
start achieving some interesting results
but one of our hosts was about
this automated wait we would find
perhaps other lost something different
than selfish fighting a different
strategy and then a type person is just
doing very poor thing
and so I never mentioned before that 60%
your producing over 90% of the boss but
what's interesting is that you're not
actually making any more money if
anything you'll be making around the
same amount of etherium and so I mean
you're just attacking the difficulty but
it's not necessarily beneficial and yeah
we couldn't find a better strategy that
selfish mining but it's interesting
because then what you can do is use
these types of systems to test different
particles and understand if there's
something going on so perhaps you can
see CNN signal detection system that
there's something wrong with the
protocol because if you don't follow the
normal strategy you're doing better and
so this can help you understand evaluate
in different ways what's happening and
you shouldn't just think that this is
not a wholesome solution but rather a
good indicator of what can happen and so
this is an open source project it's all
I can get hope so if anyone feels like
they want to test their own protocols
you're willing to do it and yeah that's
it
[Applause]
you