foreign
[Music]
this is a workshop will be
talking about using agent-based models
for analyzing D5 protocol and the
examples will be we'll be using is
through the Vega Market simulator
we do a quick introduction and then I'm
going to jump to a slide which will show
you
where to go on GitHub so that you can
hack along I really assuming it's a
workshop right so I'm expecting everyone
to have a computer in front of them and
start hacking but if not then hey you
know you can just stay along for the
chit chat so
my name is David I've been with Vega for
the last four years I mainly look after
the
kind of Financial Risk and economic
design game theory analysis and together
with me is Tom who is coming from
traditional Finance he's a Quant with
Vega help working on the working on the
research team and one of the speakers
Mark isn't isn't here but he's done a
lot of the a lot of the RL stuff so he's
the main brain behind that
so I'm going to jump to
I think here
so yeah we'll go back to introducing
what Vega is in a minute but given I
don't know how fast the Wi-Fi is going
to be in here
um just running through a few firsts
that set up steps that we can get going
with while we're presenting uh for
anyone who hasn't noticed there is Wi-Fi
specifically for the workshops uh called
Defcon workshop and the password is was
it built build it 22.
so
give that a go if the Wi-Fi's being slow
uh so this uh repo at the top here Vega
Market Sim should be open if you go over
there clone that
that's your your first step you're also
going to need go uh 19 I think it's
necessary python ideally 310 39 might
work and poetry just because it's
probably the better uh package manager
probably there's a
requirements.txt if you just need that
as well we can go through actually
setting it up in a bit
uh then if you want to be able to view
our sort of front-end console
you'll also need some some UI framework
yarn and NVM for node management and if
you want to try some RL at the end then
Pi torch is a good starting point to
download so
broadly if everyone can
make a note of that URL before we switch
back and then we will start on actually
presenting
is everyone good
cool
we'll have an actual break for setting
everything up in a bit but
yeah so we're back to the introduction
but it's it's sort of a workshop and
if you need help the person who isn't
speaking can actually come and help you
with your you know whatever it is you're
trying to do within this framework and
if you're not having luck then we can
see if one of us will have better luck
and there are not so many of us and
we've got I think two hours so we've got
plenty of time to actually get everyone
going and hacking
so in terms of what we'll cover we'll
quickly introduce Vega protocol then I
will try to make sense for agent-based
simulations in in the context of
analyzing D5 protocols
and then you know we will introduce what
you already started setting up which is
the Vega markets in which is relying on
something called vegan all chain will
um we'll explain what that is
and then once we have that we will sort
of explain
how to use the market same how it's set
up you know what what environment is
there uh what agents are there and what
you can do and at the end we're gonna
show you how you can you know train
reinforcement learning agents on top of
this and why you might possibly want to
do that
so that's that's the plan
so we start with what is Vega protocol
so Vega protocol is a layer one
blockchain for derivatives trading it's
uh set up from the ground up for trading
margins products and the price Discovery
is either through limit order books or
through auctions where do the auctions
come in typically when you're setting up
a new market you don't know what the
price is so initially there it's running
in an auction the auction determines
what the opening price should be and
then if everything goes well it's it's
running as a limit order book
if there are big price moves according
to the risk model it might switch to a
protective auction to make sure that the
big price move is really you know what's
happening economically it's not someone
with a fat finger you know for getting a
zero on an order and or adding a zero on
an order and causing causing Mayhem uh
anyone can create a market what it means
is you submit a market proposal to the
blockchain and then it goes through the
governance and it's and it's voted on so
that's what's meant by permissionless
here and what kind of markets at the
moment it's it's a supports cash settled
futures
there is roadmap for supporting any kind
of derivative through a derivative
payoff language which which will be
probably basm based but that's that's
just on the roadmap for now it's it's
cash settled Futures and so basically if
you can find an oracle
for the market then you can then you can
propose the market
and
it will run
um it's got a bespoke liquidity
provision mechanism because it's it's
not based on now ever so popular
constant product markets
but it's based on a limit order book so
you have to do something slightly
different to bond liquidity to a limit
order book so it does that and at the
moment the the assets that have value
that you would use a settlement asset on
Vega they're bridged from ethereum
though in the future that there will be
other Bridges but uh ethereum bridge is
there you can on Vega before proposing a
market if you wanted to settle in some
other asset that's not there yet you can
propose the asset the asset will then be
bridged from ethereum so you can imagine
that there will be usdc tether ethereum
whatever people want to do is that if
you're designing something like a
trading platform some of the goals that
you might have in the design
really sort of run at Cross purposes
between a general purpose blockchain and
uh you know what do you what you would
ideally like for a trading platform
so the things that I picked out are you
know a general purpose blockchain you
you need something like a gas mechanism
for people to pay for the cost that
whatever transaction the chain is
processing for them you know the cost
has to be covered by the gas
and if you're trading this is really you
know really sub-optimal in in the sense
that there are transactions that you
want to include and you don't want to
penalize them by charging for them you
know if someone wants to post a limit
order that's good they're providing
liquidity you really don't want to
discourage them if someone cancels an
order that's good they're providing
price information you probably don't
want to discourage that either
so you know that's in products they're
effectively promises right if someone
doesn't look like they're going to keep
their promise you want to close them out
immediately ideally you don't want it
farmed out to some Bots that are running
somewhere because then you're opening
yourself up to Mev and on a whole whole
host of other problems so you can have
Atomic closeouts
again for margin you want to do
potentially quiet Maps heavy risk of
computations
and you don't want to be paying for gas
for those on a general purpose
blockchain
you can try to do
Fair ordering or as Fair as your
definition of fairness allows Fair
ordering of orders and you know you you
have one specific application in mind so
you can optimize for latency for
instance rather than optimizing for you
know data throughput so you're not
fighting at Cross purposes with other
applications that would be running on
top of a general purpose blockchain
which would have other design goals in
mind
and one one thing that I didn't put on
the slide but that's also a very good
reason is that sharding is actually much
easier in the context of trading because
something happening on this market and
something happening on that market can
be completely separate apart from
Talking together through a treasury
chart so you have a very simple use case
in the sense of what needs to know about
what you can design charting relatively
easily
okay so that was that was Vega Now sort
of more General
case for doing
just trying to get rid of the mouse more
General case for doing
agent-based simulations in in the
context of D5 so
you know we've all seen various D5
protocols and uh we've seen you know
when things go well it's great and then
when things don't go well it's called a
hack or an exploit but quite often it's
not an exploit of a code bug though that
can happen it's it's simply an exploit
of of the mechanism design you know you
you go
you want to design a protocol you start
you you think what incentives should I
create so that I've got the desired
behaviors and you do it in isolation you
really try to think it through right you
you take the smartest people you can
find there are
you know dozens hundreds of incredibly
smart people in defy they do their best
they do what makes sense
they think it through as much as they
can
but uh it's difficult it's really really
difficult to think through all the
consequences especially if you have one
incentive mechanism here and other Hair
Design maybe in isolation and now
they're interacting
right
yeah
oh vendor is a name of Claus's fairness
pre-protocol for transaction ordering he
can tell you all about that he's there
the the
you can just go and show him no
yeah
yeah so you you put two incentives
mechanisms together if they were
designed in isolation it's it's not you
know completely clear that the combined
effect is actually what you've desired
and then of course D5 is all about
composability they love that but once
you introduce composability you're
introducing another disproof stuff you
set up a hypothesis which is testable
you will never prove that it's true but
you can set it up so that you run the
experiments that should disprove it if
you try their best and you've not
disproved it then you'll then you let
the hypothesis stand still
they'll prove an otherwise
and if you're wondering what this is
you've probably most of you have
probably seen Game of Life it's it's the
idea that you know you you can take a
bunch of very very simple rules and they
can lead to very very complex behavior
and sort of agent-based models are one
thing that will allow you to analyze
this complex Behavior
so yeah uh definition of what an
agent-based model is from from Wikipedia
and everyone's favorite agent and quite
often you want a lot of Agents or a lot
of similar agents so uh agent Smith is
good I've used this I've used this
picture actually twice I I was when you
talk about civil attacks I like to talk
I like to use that picture but here it's
not about civil attacks here they're
good agents who are stressing your
protocol
so in terms of you know what type of
Agents you you would find in agent-based
modeling there are the the simplest ones
are what I call zero intelligence which
is a bit insulting but uh what it what
it means is that they're not learning
and they're not optimizing you just say
this is the state of the world and if
this is the state then they're gonna do
this and if the state is something
different they're going to do that so
you know they're agents
that have hard-coded actions
and that's what they do they're the
simplest to set up they're the simplest
to understand
and of course if you hard code the
optimal actions then they're perfectly
good
if you don't then they're only as useful
as the actions that you've that you've
hard-coded
and so the next level you you can think
okay you know each agent I'm not going
to hard code the actions a priori but
I'm gonna tell the agent exactly what
the environment what the rules of the
environment that they're in
are exactly what the rewards are for
interacting with this environment
according to the rules
and I'm going to optimize in other words
I'm going to solve for the optimal
actions and that's something that comes
under you know the name either mark of
decision processes if you're doing
discrete time steps or control theory
stochastic control theory it's it's a
reasonably well established field
and you know there are a bunch of
algorithms I could spend hours talking
about algorithms that they have for
solving problems there is there are a
few problems that you can actually solve
by hand one of the most popular is the
linear quadratic regulator which
you know
um together with
the Kalman filter was one of the great
successes of of signs of the 60s you
know Blended on the moon and all that so
you know good stuff and then sort of the
the fancy new stuff is is the
reinforcement learning where you
actually don't have to tell the agents
what their environment is they just
repeatedly interact with it observe the
outcomes of their actions observe the
rewards and update their behaviors so
these are these are the type of Agents
you might want to run
you know on top of your environment to
see what is happening and of course what
is what is appropriate and what you need
will very much depend on what you're
trying to simulate what you're trying to
test but basically this is your zoo and
then the typical setup for for these
things sort of you you understand you
you have the agent agent takes action it
changes the environment in the sense
that it moves it from the current state
to the next state so sort of new state
comes out a reward comes out
and the agent takes in the reward and
the state and again chooses the next
action and and you repeat and of course
you can have you know these setups that
you repeat
sort of Forever Until Infinity maybe
with some discounting or maybe you
assume that there is some steady state
or there will be a terminal time when
the Game Stops or whatever problem
finishes then you evaluate how
everything then whether it will end well
or badly and then one thing I wanted to
point out here is because it will be
relevant if you're looking at any of the
code that we're providing is that the
agent as it learning it sort of it's in
a state it takes an action collects the
reward
sees the new state and decides on the
new action and if you record this that
that's sort of all the data that you
need to train the agent sort of offline
separate from the environment if you
collect these sarsa sequences then you
can train the agent separate from your
from your environment
okay so that's that's not so important
now it will be more useful information
later
right and I think I'll hand over to Tom
now
who will evoke you through
the you know the Vega Market Sim
um
it's a workshop so if you have questions
I think the gentleman there already said
yeah you're setting a perfect example
keep asking questions
oh do we need a microphone
you think
thank you hello
hello
um do state space
approaches have any role to play in
parameter optimization
like a control control theory based
approach for optimizing a
ffected yeah so so you would set up you
know your environment which in you know
in in the case which we will show would
be you know the whole Market how the
trades happen how the orders are matched
and then you run that whole thing
through the simulation see what happens
and then you want my want to do the
optimization on top of that right but
that's a agent-based approach right
there's you could you could use a
uh like a analytical solution so that's
let's call it an agent-based approach a
numerical approach but a control theory
approach yes if you could if you could
if you have an environment
which is sort of sufficiently simple
that you can really write down you know
all that state transition equations then
you can go and solve it through you know
either pen and paper if you're supremely
lucky or from through something like I
mean you can policy iteration value
iteration something like Matlab rather
well that's actually numerical but you
could use
um software obviously to yes yeah and
there is software out there yeah
uh does this work for creating bots
trading Bots yeah I mean like the whole
framework you have here yeah so we're I
mean Tom will talk through that so so
I'll jump a bit but basically you know
the environment on its own if it's just
the market you know it's an empty Market
it hasn't even gone out of the opening
auction if no one proposes anything so
so you automatically need bots so we
will talk a bit about the Bots that are
provided as part of the environment and
of course you can write your own Bots
uh that can do you know well whatever
they want to do we've got uh
we've got some basic Market making Bots
we've got some basic you know price
taking Bots we've got some basic
um momentum Traders
but Don will talk more about that so
I'll hand over to him
just a quick question so if
this type of systems do you take the
code from what you are testing for
instance for the Vega protocol I just
quickly check that it's it's all go code
right okay so do you take this code and
run it inside your simulation or do you
create a model of that
okay and do you think that that's that
scalable if you want to if you want to
simulate
thousands of millions of Agents because
perhaps you need you need that or do you
think it it should be
more in some cases you need a model of
what you are actually testing
the microphone so so we're running the
whole Vega stack again Tom will show
that but you're absolutely right and in
some situations especially we want to do
reinforcement learning having a
simplified model on which you can train
before you actually go on to the full
stack which will be inevitably slower
than you would like because things are
never as fast as you want
would be useful but we don't have that
at the moment we're learning on the
whole thing that's why it's important
that we use an algorithm which collects
the sarsa sequences and so then I can
learn offline but we don't have model
there are reinforcement learning
techniques which sort of build up model
from observations and you can do a lot
of clever stuff but but I'm far from
being an expert on that okay thank you
so so one thing that I didn't say is one
reason why D5 is so good for agent-based
modeling is you know you can try and
people have been talking about
agent-based modeling in the in the real
economy and economists would love to do
that but they don't have the environment
on which to execute it and in D5 we do
have that because things are typically
open source you know I can go and take
the smart contract and run things
through the smart contract I don't have
to create the environment I just say
okay I care about how this smart
contract interacts with this one I take
whatever ganache hard head I put it
there I set it up as my environment and
I pump thousands and thousands of
simulations through that I can't do it
in the real economy but here we can so
it's awesome
okay more questions
[Music]
okay are you really looking at things
like protocol solvency or profitability
of certain agents that are performing
trades or kind of what are the reward
metrics so I'm assuming they're not like
bugs but more financial kind of outcomes
so so for us it's very problem problem
specific so for example if we want to
test you know what some of the network
parameters should be we set up an
environment and and then we look at
things like you know if if I've got a
rationally behaving Market maker and the
parameter is X what is their return on
Capital
and I'm thinking well you know maybe
this is letting them earn too much money
it doesn't make sense but in terms of
the reinforcement learning guys
you you would typically look at
Financial rewards and then yeah you can
either run it thinking of it you know
maybe you're thinking they're trading
perpetuals so it's just forever or
you're thinking that they're trading a
settlement market and so the reward
comes at settlement but it's it's very
much something that you have to design
for the specific purpose that you have
in mind this is sort of more the toolbox
and I'm gonna you know wrap up the
questions hand over to Tom and then you
can ask afterwards because I have a
field at with the questions we're sort
of jumping ahead a bit so I I want to
hand over thank you
okay
so yeah discussing the market simulator
will get to doing so ourselves in a
minute but first I think it's useful to
go through and yeah to cover again
slightly more technically some of the
some of the questions how we are how we
run these simulations
um and how
how we get interesting results out so
the vegan market simulator broadly the
way that Vega itself is set up we run
with a tenement consensus layer for
those who don't know tenement is it's a
cosmos based consensus the problem with
that is that
it's generally tied to our block time
which is about a second so if you run
the whole thing you can watch the market
going second by second and you can
interact with it but
you can't run as the question in the
back said thousands of scenarios run it
through a year do all these things so
what we do we strip that out we run what
we call a null chain entirely on on one
PC and this just broadly accepts any
transactions it's sent it will it still
checks they're valid it still runs
through the whole Vega core logic but
um just adds into a block allows you to
control time forwarding so you can it'll
just sit and do nothing until you tell
it to run forwards or fill up a block
and so you can inspect at any point in
time
for however long you want
what's going on
then on top of this once we've got that
running
we have in Python an API layer which
lets you express
your trading actions your agents
behaviors in much more Market Primitives
without having to worry about the fact
that you're running on a blockchain
occasionally it still Creeps in because
that's the way well but you can broadly
talk about an agent trading on a market
without having to worry about
what Block it's in whether it's been
confirmed and this kind of thing
once we've got that we then build out
our scenarios and our tests so I'll
cover
possibly next slide exactly what a
scenario is but we can with a range of
composable Agents composable setups
build out
scenarios to investigate and test
either certain parameters test that the
market doesn't explode test this the
core itself doesn't explode or if you
just wanted to build an agent you can
try and have a realistic market and
throw yours in there
how this all works is your python layer
manages we've got two different kinds of
interfaces at the moment a Vega service
null which spins up this as I mentioned
a service entirely on your machine or
what we've recently created which is
Vegas Network and that allows you to
connect to a running external network
and run the same agents that you've just
been running locally
but on the real Network that's going on
so that can be useful in some cases too
it allows us for example on our test net
we can run agents and simulations that
other real people can interact with and
see how that's working
um we'll demo all that in a bit
um so yeah back to setting things up
and viewing
I'll go through the viewing in a second
but yeah
yeah I was gonna gonna break here anyway
so yeah so
um we're back here I don't know how many
people have managed to get to the end of
this but I think we'll probably take a
sort of 10 minute break to catch up
anyone who has them we can wander around
and see whether everyone will manage and
then 10 15 minutes we can come back and
continue with doing something more
interesting with it if you think you've
got it set up try and run these
uh scripts at the bottom we can also
tell you something more interesting to
run if if you're stuck
um I think I had a question yes
so I can then more uh in a more real way
basically test my uh my ideas it's
actually it's something we're pretty
much currently working on yeah is being
able to better snapshot
an existing running chain and be able to
totally restore from that weird there is
there's the capability currently to you
can run your simulation and save that
and rerun the same thing so you can have
a
an example that you run and then run
your things after we don't currently
have a way to pull from an existing
Network
okay thank you very much
that people should be using in their
protocol and I tend to tell them that
they can more easily solve for
tokenomics problems by including a
market mechanism to optimize and let you
know the market do those kinds of things
like example uniswap V3 solves the hard
problem of how to set the curve by
allowing concentrated liquidity so my
question is do you guys have an
intuition for which kind of simulation
like which kind of systems require a
simulation and which kind don't
oh good question I I guess I'd go back
to as David was saying earlier it
depends on how well you can if you can
solve it mathematically that's probably
always going to be optimal but when it
reaches a certain point of complexity
um yeah I don't know how you would do it
without either an agent-based simulation
or
casting modeling or that kind of thing
is there an upper limit to the kind of
complexity that you can model in this
because I'd imagine like in this example
of using a market mechanism that adds a
lot of additional complexity is that
like would that be an issue in the Vega
simulation
in terms of complexity well for the
simulation yeah I mean it multiplies the
number of agents in the in the system
um by like orders of magnitude if you
have to use a market to
um Source lots more participants
ladies I guess is there is there an
upper limit to the kind of complex
system that the Vega Market simulation
can handle
uh well practically I guess there is
enough limit we are the the system
itself we're hoping we'll be able to
scale to many thousands of people so
that should also scale to to the number
of Agents so I guess it's it's a
a goal of both the Vega protocol itself
and the market Sim thankfully have
similar things because ultimately what
we want to do is build a realistic
market so if you can build as long as we
can run a realistic Market on Vega
protocol itself which should be able to
do in the simulation cool thank you guys
something
they're exactly
going back to the going back to the
points that we were making I absolutely
agree that if you
uh if you're designing tokenomics or
protocol novel then either you can prove
that it works or you can use this
agent-based simulation to sort of
not prove that it works but show that it
doesn't and go back to the drawing board
yeah
so in in terms of training though there
are two questions I I can have the
environment sort of I can have some
agents that are better you know the
agents take their actions quickly and
get on with it but of course the moment
you have more than one optimizing and
learning agent then it's the whole you
know Game Theory and and you know that
they're probably and and then it's also
very difficult to set up or very
difficult you have to think carefully
about how you set it up you know
because is the is your is Agent a also
trying to learn the optimal response of
Agent B so that it can anticipate
Etc and then you know they can end up
chasing themselves in circles and all
sorts of weird and wonderful things can
happen you know it's uh game game theory
and NRL comes in and uh
it's it's very problem specific but you
can you can end up with situations that
they don't learn
where you can end up in situations then
they take a lot longer to learn because
there is more than one learning
can I have a show of hands of the people
who are actually trying to set it up on
their computer okay okay okay so so
we're we're sort of now is the time to
take the break and we're going to have a
have a look around
Okay so
I think a few people have it running at
least uh we're gonna I'm gonna crack on
and then we'll have another break in a
minute for
doing the next bit so we can keep going
around for anyone who's who's stuck on
this bit but
I want to give more interesting things
to do on top of that so
a quick aside once once the
the
null chain is up and running the micro
SIM is up and running you see
anyone who got it working would have
seen something like on the right here
which it is
it's a printed output log of some stuff
that's going on inside
markets him exposes various apis for
this but often you don't want to just
have to manually call python apis for
every single thing that you might want
to experience
so we've got a couple of useful portals
portals on your data you can if you've
set up the UI components as well change
a flag uh this run with console
equals true
in generally wherever you happen to be
running your Vega uh null chain that
will bring up in a web browser the nice
console that you see at the front there
that is the same console that people use
on Vega protocol itself so that has
theoretically pretty much everything you
need you can even trade manually through
that as well if you want you can log in
as your agents that you've set up
uh we'll probably demo it a bit in a bit
you also can
is actually always launched if you take
the logs
there will be this graphql port and if
you go there and you know graphql you
can query the entire system with a
fairly
hopefully clear graphql queries there's
on um what is it docs.vega dot XYZ
there's a link in a bit
the entire sort of structure of your
graphql queries you want as well
so
for anyone who has good at running for
more interesting scenario that's
something you can try out running I'll
leave it up um actually let me skip to
the next thing
this is what I'm going to talk through
with this app so anyone can type it up
so the components that
Vega itself runs on a on a validated
node that we're also going to be running
locally
that the null service spins up there are
a few different ones we've got Vega
which is the core process that's the
um go
blockchain client that does all of your
transaction processing and outputs your
States
that is obviously it's a point in time
proceeding thing so we also want the
data node which is listens to external
events output by the note core node
itself and saves those into a
a postgres database and allows you to
query all of that
historic data
and that's generally what you'll be
using when you're querying any data from
Vega
storage layer additionally there's a
couple of extra things there's a console
as I mentioned previously that isn't
necessary but it's very useful front end
there's also the Vega wallet
the wallet is basically size
transactions and ensures that
any sort of custom signing we have some
proof of work to stop spam attacks and
stuff and that handles all that it's
optional here because for the null chain
what we actually do is Skip all of our
validation so that it will blindly
accept if you send a transaction and say
it's from whoever it believes it's from
whoever so you don't necessarily need
that what it will spin up is if you spin
up a console the two interact and a lot
together so we use a full wallet for
that
so
once we've got our our Market Sim set up
how do we actually run our scenarios
there set up as
ideally Plug and Play components the
overall
thing we set up is we call a scenario
that consists of an environment which is
your background environment and your
Vegas setup so that will include things
like how many steps you want to run your
simulation for
uh how your Vega is set up
and sort of what kind of logging you
want that kind of background
detail we then plug in the various
agents some of whom will be setting
prices they could be acting randomly and
each of them will have
something they do you'll see when you
build one yourself in a bit something to
do with startup often they'll force it
themselves some tokens that kind of
thing and then they've just got a
function and each step call that
function and it does whatever it wants
to do it pulls in data from the market
it trades they do whatever
so scenario environment
you'll
see that when we go in a minute to the
agents themselves so yeah what's an
agent it's it's fairly simple ultimately
it's an initialized step where they
generally force it sometimes they will
um do things so we'll have an agent that
sets up all the assets because you start
with a blank blockchain you don't have
any assets any markets that kind of
thing so there's you may see if you look
through a market manager who does all
this setup for you then they'll have a
function that's called every scenario
step however many steps you do generally
will sort of randomly shuffle them so
that you're not always getting the first
agent called First in any step
and then a finalized method that most
mostly you can ignore but uh often
you'll want one to sort of settle the
market and things so that's how you you
want to set up your agent there's one
more thing I mentioned earlier that
sometimes the blockchain does have to
bleed back in you'll see scattered
throughout occasionally you'll wait for
total catch up function basically what
this ensures is that you're getting a
good view of the market currently you
haven't
um thrown way too many
uh transactions at it everything's in
sync that kind of thing so it just
ensures that your more real world
modeling rather than trying to ensure
that you can get everything in
one nanosecond through
so
um
I'm gonna now talk through building one
of these agents a really simple one it's
just gonna do a bit of training I guess
and
then we'll break again and hopefully you
can put together your own a bit
where you're
thank you Simmons
Okay so
for anyone who has it set up we've got
this run simple agent in
exam reinforcement run symbol agent I'll
put up on the screen after this
to pass to these files
and then in agents
we have
our simple agent so what run simple
agent does is it sets up a scenario
um I can discuss the actual arguments
with people but
which we call curve Market maker
and that
a
H jump into marketing basically that
contains an agent who ensures that
there's a nice shape of the market and a
few who just sort of randomly buy and
sell so they're very dumb theoretically
you should be able to
agent which is what hopefully we're
going to build
that runs
this simple agent that we've got
together here so
what I've put together is a few
nice template functions that is without
having to worry too much about any of
this works you should be able to cut and
paste these into our step function
which is down here
we've got the simple agent
and you can see it's got the structure
as described earlier we've got our
initialize where it's finding the market
finding the asset and minting itself an
asset
and then say
submit the market order
from that
check her positions
is it almost gonna buy
it is currently always going to buy so
50 50 chance of making money
Okay so
uh they're down to the relay just the
hardcore Workshop participants you know
you're the ones who prevailed with us
despite Python and its foibles and the
slow internet
so I'm going to crack on if you're if
you're coding keep coding uh you know if
you have questions there will be time to
ask questions and what we went to what
we want to get to now is
just to show you a little bit more of
you know what the agents that that you
have there you know the market maker Etc
that you're interacting with a little
bit the ideas that they're based on and
then move on to the reinforcement
learning part so
you know as part of the environment as
Tom was saying if you just set it up
there is nothing it's it's the empty
Vega blockchain as it would as it would
be when it launches live you know no
assets no markets everything has to be
created through governance
and you know once the asset is created
once the market is voted through which
you know the python sort of expedites
for you you will still have just an
empty market right no liquidity on it
nothing no activity
so
effectively the code provides various
versions of of market makers that are
based on some stochastic control
problems that are pre-solved coded up
and basically running and then some sort
of hacks where you know the optimal
solution of the stochastic control
problem is used as a heuristic and then
you put something on top of that
and you know there are you can you can
have liquidity Trader you can have
liquidity takers it just means they
Place random Market buy and sell orders
you can have informed Traders what does
it mean well you just tell them what the
price will be at the next step right you
know maybe they they have inside
information or whatever
and um
there are the momentum Traders through a
library so you you can uh you can trade
on on all the patterns that these
chartist people have you know there are
these vomiting camels and I don't know
what else you know you you can code up
agents that trade on that