Algorithmic Trading and Machine Learning

okay thanks Costas and thanks for having me and I want to thank Eric for his excellent talk because it also helps set the context for a lot of what I'll be discussing here you can think about much of what I'm going to talk about today about what it's like to sort of live on the other side of these exchanges and to deal with hfts and other counterparties as well I also should say that there's really going to be no explicit game theoretic content in this talk which Costas promised me was okay but there will be quite a bit of practice as many of you know I for the last 12 years or so I've been working with a quantitative trading group on Wall Street in my sort of shadow life and I want to give special thanks to my trading partner you know by vaca who's a co-author on all this work all this work kind of came out of some proprietary commercial context in our training group and the parts of that I'm talking about are sort of the non-proprietary parts that seem scientifically interesting to us over time and so you know this slide sort of puts a lot of this is kind of just a context slide for a lot of the things that Eric said we certainly live in interesting times on Wall Street both from a technological perspective and also more generally from a social perspective so here's the same image that showed of the flash crash a few years ago here's the famous book by Michael Lewis which I highly recommend on high-frequency trading we have Bernie Madoff other insider traders and the like and so without passing moral judgment on any of this activity including high-frequency traders what I want to do in this talk is just lay out what some of the algorithmic challenges are in trading in modern electronic financial markets from the perspective of somebody in working in a quantitative trading group essentially something like a traditional stat our equities trading group what we do in my group is trade equities long and short we trade pretty much every liquid instrument in many many markets both domestically and internationally then we hedge exclusively with futures so we're not doing any complicated derivatives or the like so we're in kind of the simplest markets or the simplest instruments but as Erika sort of already outlined today a lot of the complexity of trading those environments today comes from the automation of those markets and the data that's available and what you do with it and what your counterparties do with it as well and needless to say because of the rising automation and availability of data on Wall Street's there are many many trading problems that one encounters that invite a learning approach a machine learning approach right because you have the data it's at a temporal and spacial scale that you can't understand as a human being and so one has to develop algorithms to trade sensibly and those algorithms have to be adaptive and trained or should be trained on historical data sometimes a lot of historical data including very very recent data and so what I'm going to do is just try to give a high-level talk where I quickly outline three sort of little vignettes or case studies of specific problems that kind of arise in trading algorithmic trading in modern electronic markets and sort of what the challenge is those problems present for kind of traditional machine learning are and some kind of hints about how you can address those problems with new techniques and so the first two problems I'm going to talk about the first ones probably the first two are quite closely related to sort of the world that Eric was describing are both about problems of optimized execution where a person or a model or an algorithm has made a decision to execute some tray to like buy you know a hundred thousand shares of Google or short you know a hundred thousand shares of Yahoo what have you and now the problem is actually executing that trade because a directive like that is under specified in particular it's under specified in its temporal aspects right so you use you know I can say I want to buy a hundreds of thousand shares of something but how quickly do I want to do that okay and so as we'll see that there's this trade-off in all electronic markets and by the way I don't think any of this would go away with something like the very sensible alternative mechanisms that Eric is proposing you have this trade-off between immediacy and price you can be in a hurry and impact prices more and get worse prices but you might want to do that because let's say you think that the informational advantage you have is fleeting and you want to execute the trade quickly or you can try to do things in a more leisurely way and try to let the market come to your preferred price but then it'll take longer in general okay and so the first two problems I'm going to talk about are sort of very specific instances of that kind of immediacy versus price trade-off that you see in electronic markets the last one is a little bit more on kind of algorithmic versions of kind of classical portfolio optimization mean-variance optimization the kinds of problems where you want to hold a diversified portfolio that you know kind of maximizes your return subject to some risk considerations or volatility considerations okay and I may not finish all this but I'll hopefully at least say a few words about all these topics but I think I did save some time because Eric very nicely laid out kind of what the modern continuous double limit order auction looks like this is what's sometimes referred to in financial circles as Mike market microstructure and and so let me start off by just describing the canonical trading problem we already mentioned it but let me make it a little bit more formal suppose that we have a particular trading goal like we want to sell the shares of some stocks buy or sell a symmetric but just to be concrete let's say that we want to sell a certain number of shares some volume that I'll call capital V and I also have a specified time horizon in which I want to execute that trade so in I want to sell the shares in tea time steps and of course if I'm selling I want to maximize the revenue from those shares and if I'm buying I want to minimize the expenditure for those shares and as you might or might not imagine there's sort of a very large number of benchmarks that have been proposed for how to measure the quality of such a trade so if I actually go out in the market and execute an order to let's say you know buy a certain number of shares of Google in a specified time period we have to ask well you know how good how well did I perform right and almost all of the sensible measures kind of measure your performance relative to the activity that was going on in the market at the time that you executed your trade so a very typical thing would to do would be to say let's say I started this trade at 10 o'clock in the morning I executed it over the course of an hour how well did the actual average share price that I got in my execution compared to let's say the volume weighted average price of the stock during that same period that's called the V whap there's something called the T whap I'm in this particular case going to use something that's called the implementation shortfall which is you look at the midpoint between the bid and the ask at the time you started your trade and which of course by the way is itself a fictitious benchmark because by definition there's no liquidity at the midpoint between the bid and the ask but it sort of seems like a fair peg point and then I compare my performance to that and and and ask you know how much worse I did than that it will almost always be worse on average than then the midpoint of the dead-ass because you know you know in all these problems your trading activity itself is going to push the prices in the direction that disfavors you okay if I buy by buying a large volume of shares it's going to push the price up if I'm selling it's going to push the price down and this is entirely do go it's it's at least largely due to just the mechanical impact I have on the order book right as I cross the spread and eat liquidity from the other side it you know it makes the successive prices worse and worse okay so whatever metric you look at it's very if you if any of you thought about this problem seriously for just a little while you'd come to the view that a very natural framework to think about such problems is essentially problems of state-based control okay where there are at least two very very obvious state variables you want to consider how much inventory you have remaining ie how many of the V shares do you still have left to buy in the given time period and how much time is left okay and you might also want to think about other features or state variables that capture other kinds of market Civet e market activity that might have informational advantage as well okay so this is just a snapshot sort of showing what a typical order book looks like this is from many many years ago in Microsoft on-on-on Island at the time which has now been rolled up into some larger exchange but you see the buy orders arranged but in order of decreasing price the sell orders arranged in ascending price by definition there's always going to be a gap between these two which is the bid-ask spread that Eric was discussing there's volumes at each of these and just to remind you that the way these markets work is you know if I'm trying to let's say buy shares I can you know I can I can place a limit order and that limit order will either you know sort of sit somewhere in this price ladder or it might actually cross the spread and execute against the existing liquidity on the sell book okay and it might partially match so if I put in an order for I don't know let's say a thousand shares at you know twenty three seven seventy nine nine I would execute nine hundred of those shares the first hundred at this price the next eight hundred at this price and then the my remaining hundred shares would down become the new top of the buy book okay so it's a very mechanical process and as Eric pointed out it's extremely volatile these days it's extremely dynamic the sort of time scale at which orders arrive and executions take place is you know often at the sub millisecond level in highly liquid stocks or even the microseconds and as he mentioned also right you know everything is possible I can cancel I can revise my order partial executions may occur and so you know one very interesting very difficult question or type of research to ask is how do individual you know sort of microscopic orders influence aggregate macro behavior you know of the overall aggregate market but from the purposes of a trading group like our is one of the things we care about most is this trade-off between immediacy and price so you know to just you know I won't explain this chart in any detail but what it's showing you the x-axis is sort of you know what my limit order price is let's say if I'm buying right and at the left extreme I'm sort of considering prices that are sitting way down in the book so they're very very far from where the market is currently trading and so what I'm showing you here is the performance of a very simple class of strategies for this problem of execute v shares and tea time steps which is what I call kind of submit and leave strategies okay and in a submit and leave strategy what you do is you just pick a fixed limit order price put all these years at that limit order price hope that they get executed but any volume that's not executed after the T times to share steps you have to cross the spread and eat the beat and buy the shares at whatever prices they're offered on the other side of the book okay because in all of these you know it's it's it's very as a practical matter it's very much the case that whenever we execute a trade we always set a time limit on it because we essentially have some model or empirical measurement historically of how long our informational advantage lasts right so if an analyst upgrades a stock for instance there's some informational advantage in trading on that activity on that upgrade very very quickly but you know after five minutes it's gone because the markets essentially already incorporated that news okay so what you see here is that there's this very distinct peak if I put my orders very deep down in the book then nothing gets executed I have to cross the spread all at the end and pay the prices on offer at the other extreme I could cross the spread at the beginning right and pray pay the prices on offer at the beginning these two things are roughly equal but you can see that there's a great deal of optimization there's sort of a magic price at which I should put my limit order if all I'm going to do is leave it there and then cross the spread with the remaining inventory at the end of the time interval okay so this already shows you that even within this kind of brain-dead class of strategies there's actually quite a bit of optimization to do right there's a very very sharp peak and this is quite this is quite typical I don't remember what stock this is in yeah she should know so in everything I'm going to describe here I'm sort of only accounting for the mechanical impact and the mechanical impact is the only thing you can kind of optimize for using historical data right because by definition you can't ask the counterfactual question of what would the psychological or if you like you know the sort of non mechanical impact be I mean you you can have models for it and there are many models proposed in the literature but for from the viewpoint of a group like ours I mean we don't even bother looking at it in those models because we know that at the end of the day the only way you're going to measure that is by actually starting to trade right and so of course we do that in small volumes at the beginning when we're trying they knew and gradually ramp it up as we get some empirical sense of what that impact is like and as you can imagine right I mean all these things have to do with liquidity it's much easier to hide your activity and have it have lower mechanical impact and kind of counterfactual impact in a highly liquid name than what than something that trades very infrequently okay this is for some specific stock for some specific period but this is extremely generic right if you go to any reasonably liquid stock you would see that there's really and you'll exact the exact location like this zero here means kind of putting your order at the current bit or the ask so being you know kind of putting your order not crossing the spread but not sitting down too far right so because you know in a highly illiquid stock if you sit down too far on average you're just never going to get executed because you know even if a little bit of liquidity gets eaten away other counterparties are just filling that vacuum behind it okay so it's there it's always you know modal and it's and it always has a relatively sharp peak exactly where that peak is might depend on the stock in question but it's a very generic empirical figure okay so you know reinforcement learning which is essentially the AI term for you know discrete state-based control is sort of a perfect fit for solving this kind of problem where as a first cut what you might do is just say alright I'm going to keep track of two things the amount of inventory I have remaining and how much time I have remaining and whatever period I've specified over which I want to do the trade and I won't go into it but we did you know many years ago this sort of large-scale empirical study where we apply kind of fairly generic reinforcement learning using historical data to learn kind of optimized policies using those two state variables and you know the kind of takeaway message here is you learn very sensible policies and this is what you want right I mean you kind of want you know in general I mean we're kind of the opposite of blackbox quants any time we use machine learning in a trading problem or in in some kind of strategy model if we can't back out in our heads some reason of all kind of economic rationale for why this makes sense we're extremely uncomfortable deploying it in actual trading and so what you want to kind of see our pictures like this I'm showing you the actual policies learned over three relatively liquid NASDAQ stocks where the state very the state space is sort of on the XY plane where you have a certain amount of time left I've discretized the variables here so this is sort of decreasing amounts of time left in your interval and decreasing amounts of inventory left in the size of the trade that you're trying to execute and basically the the sort of z axis here is measuring you know whether you're you know kind of sitting further back in the book versus crossing the spread so the higher the value means the more you're kind of crossing the bid the higher the price you're buying right you're trying to cross the spread and you can see here you know very sensible behavior as you're running out of time and you have a lot of inventory left then you better get going and start being more aggressive in your trades if you've got a lot of time left and very inventory very little inventory so this might have happened because you know at the beginning some other counterparty kind of cross the spread and executed a lot of your shares very early in the interval well now you can afford to be patient right now you can afford to drop down in the book and hope for price improvement and get a better price okay so you know so if you do this across many many different stocks and then over many many different historical periods you will always get kind of a shape that looks like this but the details really matter here right the numerical details really matter because different stocks have different bid-ask spreads you know the this sort of price you know the bid-ask spread determines how big you have to be um you know what how large your price has to be a you know above the bid to actually reach the buyers or the sellers on the other side etc okay so this is all kind of good news yeah these deadlines an example would be like we said the analyst yeah yeah and these and these volumes and you know sort of to talk about the practice of a little bit more these volumes and these time intervals we would kind of self impose them now we might optimize them right because everything here is a moving part that interacts with everything else so for instance and and this is the kind of thing we're always trading off you know we might know that we can kind of on average get better prices by taking a longer time interval but then we'll also know there's a you know essentially a decay of what you would call our alpha on Wall Street like our even for our informational advantage right and so we will always be trading those two things off and testing them on historical data to the extent that we can and then it's better then at some point you have to you know we have to either you know if we're using kind of direct market access algorithms or if we're trading through a prime brokerage we have to have a directive right you you kind of can't say to the brokerage or you definitely don't want to say to the brokerage oh you know why don't you buy you know some Google shares over whatever amount of time you feel like it right we have to actually give them a very specific directive that some more complex would be some kind of learning model where as this is happening you might detect things at once that will make you want to change your time limit right potentially potentially yeah yeah yeah and and by the way I mean like one form of that for instance is it's quite often that one aunt when one analyst upgrades their view on a stock other analysts upgrade their view on that same stock in quick succession and so one of the things you need to decide there is like is that actually fresh news or is this just kind of an artifact of some other more basic news that's getting into the market and then you might want in that case you might not even want to trade on it at all yeah why at the end of you here you want to buy the rest of the volume why not just give up time for questions I mean the high level answer okay so there's two answers to this one is if you're an actual brokerage like Bank of America and you have an algorithmic trading desk you're performing these actions on behalf of a client that's given you the directive right and so you don't ask well you know hey I didn't get all your shares of Google do you sure you still want to buy them they you know the language in which they communicate with you is as a directive to buy this many shares in this amount of time from our standpoint we do have that option but the reason we don't is because at some point we've kind of optimized all of these things together and realize that you know this is sort of the right amount of volume to buy based on this informational signal and by the way that volume is sort of usually the maximum that we think we can get away with without pushing prices around too much so you could do something like what you're suggesting but I think you know as a practical matter we you know there's sort of so many things to think about in this world that the fewer moving free parameters you can have the better yeah the coz of factual maca typing you know it seems to be a big problem in my mind when you when you do the testing on the six months following are you doing that live right that's still you think the model is still using so this is you know you can think about this study is limiting the sort of scope to the improvements you can get on that particular kind of market impact okay and in general you know as a practical matter the way we deal with the second kind of impact is by you know I mean first of all we have many rules in fact we even have like contractual regulations on how much of all you know sort of how many how much volume we can trade in any particular stock okay so for instance my specific group you know we will never be more that I mean we have a contractual limit that says we will never constitute more than one percent of the average daily volume in any single stock but we don't even ever want to get close to that we will we will sometimes be a half a percent and what by the way we're not an especially large group right so there are a lot of groups with this sort of behavior that they're willing to be that were you know like a quarter of half a percent of the average daily vol yeah so I'm very happy to see that you're talking about a glass box rather than a black box much burning and I guess this being able to for human beings to be able to understand what is going on I wonder given the complexity of what's being learned the special tools you have to explain these policies to people is that more done by eyeballing I mean in the case of a problem like this it's very clear what sort of sensible behavior means right if I somehow the learning process told me oh you're you're running out of time you have a huge amount of inventory left and so you know reduce your buy price lower this is kind of obvious the harder thing is when you have strategies that work well and you don't quite you know work well historically maybe they are even working well in live training and you don't have a clear handle and and you know and so here I mean I think this is the nature of the problem makes what's sensible clear other times you we find ourselves admittedly kind of thinking about almost thinking about it from an anthropological perspective which is like you know if this trade is consistently making money there's got to be some other group of counterparties out there that are you know are taking the other side of this trade and what you know what is the reason you know what is their reason for persistently doing this you know other other than you know stupidity right and actually you're worried if the other if the reason is stupidity right because people real you know people often realize that they're doing something stupid and stop doing it and then you can lose a lot of money but if there's sort of a good reason why I mean so like to give a concrete example it's very consistently you can very consistently make small amounts of money by in the period before the market opens laying down a ladder of limit orders on both side of the books in every liquid stock but spaced far enough that if your first order gets hit it's a significant kind of opening you know ten minute deviation for that stock okay and if your second order gets hit it's like a you know an even larger standard deviation so basically just laying down these orders on both sides of the books seeing if anything gets hit and then liquidating it over let's say the first two hours after the market open very consistently makes money it's not easy to implement this in kind of you know there's a lot I'm the reason I'm telling you this is that I know that the engineering details of this are non-trivial but but so the reason why because so why is this well the reason is that many parties that sort of trade on a slower time scale like mutual fund managers first of all they like to trade at the open of the close to try to benchmark to opening and closing prices because that's how their performance is benchmark a and B right there often reacting to overnight news right and so they're all coming in the morning but but it's sort of worthwhile for them to do so because they want to get it close to the opener to the close the same thing works on the close as well so there it's not sort of like there's an obvious optimization criteria and there's a sensible behavior to it but it's there's more of a sociological exclamation to it okay so now I mean already you know again just on this aspect of mechanical market impact if you compared you know sort of how well you can do by keeping track of these two simple state variables and taking kind of a control theoretic approach versus let's say the best you could have done by optimizing within this class of submit and leave strategies right which don't know about time and remain volume remaining they just keep an order fixed and then when time runs out they cross the spread you can all you're already getting 35 percent improvement on average using this kind of approach compared to that and this is a meaningful improvement in this world because you know as think as Eric mentioned right you know kind of on this particular problem performance Delta's like sort of you know the the sort of meaningful improvements in performance are measured in basis points which are hundred hundreds of a percentage point right so if you're sort of getting thirty five percent improvement over something reasonable it's non-trivial yeah so there's something to leave here in comparison is measured without some Italy price is optimized really high side yes so it's like a learning to learning experiment so either do reinforcement learning on the historical data or use the historical data to find the best single limit order price within this restrict class of strategies you already get a thirty five percent improvement but the reinforcement learning is that also giving insight going on so it's all train test methodology we take a period of historical data we train reinforcement learning learn some policy on that historical period over the same historical period we find the optimize submit and leave strategy we compare both of them on this excessive out-of-sample time period you're getting a thirty five percent improvement one days like information chained to the market or you can be like you're saying basic like any arbitrary interval okay so so I mean these experiments we did over a long historical period with many many kind of sliding windows of training and testing data to get you know to get into the details a little bit one that one of the things that's hard in applying machine learning to financial or market data is the fact that if you're not careful you can learn unintentionally all kinds of like you know just sort of drift okay so if you happen to train over a period in which the market is just rising overall you know you'll learn a policy that you know basically just says you know you should you know you should buy right because the market is rising at a high level right and you don't want to learn this kind of drift you can also you know learn other sorts of seasonality x' which by which I don't mean you know fall/winter but other kinds of you know different trading days have different volumes different periods of the day have more or less trading activity and so you want there's a lot of engineering that kind of goes into kind of creating your models in a way that make sure you're not just learning these seasonality x' or these you know kind of directional drifts okay so you know it's of course natural to ask once you see that this sort of simple control theoretic state-based approach works you know reasonably well to ask well maybe I should add other state variables which don't just have to do with how much time and volume are remaining but that are actually looking at you know what's going on in the market and since we're in kind of a double limit order book Oh structure setting it's quite natural to look at features of the order book itself right or the recent temporal activity in your book and by the way I am of the belief or kind of semi knowledge that I think a lot of hft firms who are not trying to solve execution problems per se they're actually trying to predict directional movement in the market right but I think they're using a lot of the same kind of machine learning techniques I think they're spending a lot of time sitting around inventing features of the order books that might have some slight informational advantage and let's say predicting whether the midpoint of the spread is going to go up or down in the next 500 milliseconds for instance okay it's not easy work and it requires sort of technology that's not really in our domain so we don't do that kind of thing but I think the methods are very much the same and I won't go into this I think so yeah I mean I think especially at the micro structural level you kind of have to be right because there's no I mean at some level it's not you know there's you you can't really understand it right it really has to do with sort of volumes and speeds of data that are beyond kind of normal human comprehension and so you mean you can be very careful about your train and testing methodology and sort of know that's there's some consistent predictive power to some state variable but knowing Y is very very hard at that granularity right so this is just showing you a list of a bunch of kind of features of the order book that we invented and add it to this basic state space of time and volume remaining and the percentage improvement that you get by adding them I'm not going to go into all of them but you know a lot of them are quite straightforward like what is the bid-ask spread right so knowing what the bid-ask spread is at a given moment is a very valuable thing to incorporate into the state space because in particular you know implicitly now the policy can learn exactly how high a price it needs to put in to just get to the bid and not let's say overshoot the bid by a lot right because if I basically say like well I'm going to raise my current price by 15 cents like well it is that 15 cents just reaching the other side and eating a little bit of volume or is it actually crossing very deep into the book and giving me terrible terrible prices things like signing trades is very useful signing trades is basically going through the order books and saying whether each trade was initiated by a buyer crossing the spread to the sell book or a seller crossing the book - the Bible so you know and some of these things right actually don't help you that much so we thought it would be helpful to know exactly how much volume there is at the bit or the ask it turns out this is not I mean it looks slightly harmful but it's basically a wash it doesn't seem to have much informational value negative actually means that you know in the experiments it's slightly hurt performance to add that feature to the state space and this could just be for overfitting reasons right like I if I add some variable which doesn't have any informational value then you know my model in training will you know fit kind of to the behavior of that variable but it's not useful out-of-sample but you know it turns out that you know if you take kind of the best of these features that did add informational value and combine them you get kind of another thirteen percent improvement over and above the thirty five percent improvement that I described on the last slide okay okay good so you know just again to kind of riff or freeride on Eric's talk one of the things that's kind of very interesting about modern financial markets and their evolution over the last 20 years or so is that you know again as he kind of intimated sometimes in kind of going to automation in apparent and transparency actually there was some kind of human aspect of prior trading that served a very useful purpose that was eradicated and you know it made certain types of trade much more difficult and then some new electronic mechanism was introduced to try to replicate that functionality before and so an example of that is you know again you know this picture of a double of continuous double limit order book this has been around since sort of the dawn of financial markets it's just that prior to automation on Wall Street for instance you know it would actually be maintained in a paper Leger by the market maker in that stock who would sort of you know have these little shits of paper and order them you know in order of increasing or decreasing price whether on the buyer of the sell book and and so that we're going to do there wasn't this kind of transparency but with this kind of transparency where this data is freely available to you know anybody who wants to can do you know real-time reconstruction of limit order books and see kind of the trades of everybody else not not earmarked by identity right you're not you're not being told oh this is a credit suisse order and this is from sa C capital or something like that but you can definitely algorithmically detect for instance when there's a large trade in the market when somebody's trying to let's say unload a lot of some stock in some small amount of time you know the way they have to do it is through a market like this and they're going to leave an algorithmic trace of their activity in the data okay so in the old days when let's say I'm you know suppose there's some mutual fund manager who's you know life goal is to track the S&P 500 okay and you know because of because of you know volatility in the market you know let's say for instance Johnson & Johnson has done extraordinarily well and now because Johnson & Johnson has done extraordinarily well they're overweight Johnson & Johnson right they're out of alignment with the S&P 500 so they need to unload some Johnson & Johnson and they're like a fidelity or Vanguard fund manager so actually their portfolio is so big that they let's say you know they have 1 percent of the outstanding shares of Johnson & Johnson a huge stake in a public company and they need to reduce it to you know like 0.8 you know 0.8 percent instead okay this is a massive massive trade and the way the mutual fund manager would have done this prior to automation is they would have it would have been sort of an off the floor or what they would call an upstairs trade they would actually call you know a somebody at a brokerage they would call one of their buddies at a brokerage or multiple brokerages and they would say look I've got this rather difficult trade I don't want to execute it on the floor of the exchange because it'll just be too obvious what I'm trying to do and you know prices will move against me you know could you possibly shop around and see if there's a counter maybe there's somebody at maybe there's somebody else out there that happens to want to buy you know 0.2 of a percent of all Johnson & Johnson outstanding shares so that we can kind of there would still be a record of it eventually at the end of the day or the end of the trade it just wouldn't have taken place on the floor of the exchange would have taken place by people making phone calls and saying okay I found a counterparty for you at this hedge fund or this other investment firm okay that kind of got eradicated when the world went like this and all the activity had to be executed through these books in electronic fashion in order to get competitive prices and this was sort of the initial impetus to what are called dark pools okay so what are dark pools dark pools are another very recent kind of electronic exchange I think the first one's kind of probably came out about 10 years ago but they became quite widespread about five years ago they were essentially they were introduced to sort of deal with this thing that went away that I described to allow large counterparties to sort of trade with each other at minimal market impact okay and the interesting thing about dark pools is that unlike the double limit order book you don't specify price and volume the way you do in a limit order you just specify volume you just say this is the volume and I'm and whether you're a buyer or seller you're not specifying the price and so now it's sort of buyers and sellers instead of it sort of being ordered by price or instead ordered by time of arrival so you just have these two queues of arriving buyers and sellers you just line them up in the order that they arrive as long as there are parties on both sides you just keep matching them together okay and now the question is like well if they're just ordered by by arrival time and with there's no price specified when I match a buyer and a seller what price does that transaction take place that it basically takes place at the the the midpoint of the so-called NB bo which is essentially the midpoint of the litt double limit order book market in the US okay so this is the whole point here right you don't go to a dark pool looking for a price improvement you don't go there with the thought like oh I'm going to actually get a better price here that I could get by trading on the open limit order book you go there because you're perfectly happy with the prices offered on the lid market you're just worried that you would never be able to get them because you'd so quickly push the price in the wrong direction because of the volume of your trade so you're sort of have these this exchange now that's pegged to an existing exchange the so called lit market these are called dark right because you don't really see you don't see any of the activity you just put yourself in this line and you know you're just told in each second you know nothing yet nothing yet nothing yet okay your trades executed and here's the price that you've got and that price should be pegged to the the lit market okay so it's you know this was such a popular idea that there are now dozens or you know many dozens of dark pools in the United States and they're essentially you go there to compete for liquidity to get liquidity not to compete for price and so this you know generates kind of another interesting kind of optimized execution problem we're instead of trying to optimize the performance of your trade over time as in the problem I described before you're essentially trying to break it up over space or more precisely over exchanges okay so in other words the the problem that's known as smart order routing on Wall Street is you have these multiple exchange let's say you have K different dark pools and you have a certain volume of let's say Johnson & Johnson shares that you want to sell okay and at each time step you need to decide well of my remaining inventory how should I charge a distributed among these multiple exchanges to try to maximize the fraction of my remaining inventory that's executed at each step right so it's again a problem of an you have a well specified trade I want to buy or sell the shares of something but now instead of trying to get the boss the best possible prices I'm just trying to as quickly as possible unload this inventory by dispersing it or routing it to these multiple exchanges okay so we thought about this problem from a theoretical standpoint and that led to kind of an interesting algorithm a solution that we evaluated experimentally I don't have time to go into the details but you know here's one simple cartoon model of liquidity right so these so maybe I should have stayed this explicitly different dark pools will have different sort of liquidity properties at different times and for different stocks okay so for instance it's quite common when a new exchange starts whether it's a limit order book or a dark pool that they'll try to actually get a foothold in the market by offering preferential treatment or rebates or fees to some particular class of stocks so you know you will actually see marketing materials that say well you know if you're trading mid-cap consumer durable stocks you know we are the dark pool you want to come to that's where people who want to trade consumer durable stocks are coming and so this is where your counterparties are don't go to other dark pools because you'll just be sitting there waiting for your trade to get executed whereas you know in these stocks the action is you know if the action is at our dark pool okay so you know one kind of cartoon model of this is to imagine that at each times that there's some underlying stationary probability distribution for a given stock let's say Johnson & Johnson the different dark pools will have sort of different liquidity profiles meaning and and and so what I mean by this is that let's imagine that on the x axis is number of shares okay and on the y axis is sort of probability that that number of shares is available for execution on the other side at each given time step so more formally what I have in mind is that you know if we submit the shares of our trade so let's say this particular dark pool that what happens is that we draw a number according to this distribution we draw s according to this distribution okay and that's sort of the the volume of the counterparty on the other side in this dark pool at that particular discretized time step and so then what gets would get executed of course is actually the minimum of vns right because you're never going to execute more than you asked for you might execute less so if I submit an order to this dark pool to buy a hundred thousand shares and s gets drawn and it's a million then I'll just be told your hundred thousand shares were executed okay but if I submitted a hundred thousand shares and s was the desk that was drawn was fifty thousand I'll be told half of your order was executed half of it's still waiting in line okay okay so yeah so here you can partial execution but how can that curb then be non decreasing um this curve is just showing you you know so this is a curve showing you know modeling the idea that you know volume is kind of in this range are quite likely to be available okay whereas you know larger volumes are not available right if you do partial execution should it get a machine curve either so the partial execution is sort of on my side right so I submit a volume a number is drawn which is the volume available on the other side the minimum of the two quantities is what's executed and then I still have that remaining inventory that at the next time step I need to try to execute yeah I thought you were asking more of a mechanism question than a math question it's a density it's a density not a it's a you know PDF not a CDF yeah Kevin stable for there to be so many jar cool isn't it everyone's interested these are good questions I mean I'm sure Eric has thoughts on this and I'm not sure I think Ramesh isn't here but I know he's thought about kind of these questions why is there a mean it's the same thing with you know it's the same thing that happened under the sort of you know kind of regulation of financial markets there's this famous kind of incident called or you know kind of regulatory ruling called Reagan and NMS which is national market system and and I'm not sure it's an equilibrium but there does seem to be this proliferation and they compete with each other in various ways I mean it's a good question whether or not at some point this will shake out and you know there will only be one dark bull or only a couple this hasn't you know this hasn't really happened in lit in in sort of the continuous double auction either since they sort of I mean by the way it used to be the case that regular too early you know your stock could only trade on the exchange it was listed and this was sort of the golden ear of specialists and the like so if your company listed on the New York Stock Exchange the only place to buy or sell shares was on the floor of the New York Stock Exchange so when this got deregulated and basically anybody any startup if you like with the know-how and the ability to get the regulatory approvals could open up shop and say hey we're running a double you know is I mean I think to Eric's point it's a very interesting time because the regulatory landscape is such that you know startup companies can you know propose brand new mechanisms right like the one Eric's proposing what you're doing here is the endpoint the only ones are for the trends in it you know from a from a conceptual standpoint I want to agree with you but so many things in the actual world of financial markets are obviously out of equilibrium in some sense and so you know how to explain that it could be not everybody's doing what I'm proposing here that they're doing it in different ways and by the way I mean to you know the I think on this question is sort of what does game theory have to say about financial markets I think it's very difficult but at a high level you know we think all the time one of the things we're worried about is how many other people or groups are doing something like what we're doing right because any given trade has a certain capacity right you can't sort of execute arbitrary volumes so if we're all doing the same thing then our respective piece of that trade or its profitability is definitely going to decline so we don't enough time we have it understand so let's take further questions all right so let's how much time do I have is it actually zero or let me just finish this one vignette and I'll skip the last one and then we can we can plug it so it turns out that you know in this simple formalism that I just described if you kind of specify it there's actually a very simple and nice algorithm that you can use so so first of all the sort of real machine learning challenge that comes up in this model is the fact that you're your observations are censored by your own actions right so it's this thing that if you submit a thousand you get more than a thousand you don't know actually might they're more you know a million and you want to kind of know that because you're deciding how to distribute your order over these multiple exchanges and if you just ask the question what well suppose I wanted to estimate these probability distributions from my censored observations there's a well known maximum likelihood estimator for this which is called the kaplan-meier estimator or sometimes called the product limit estimator but we have an exploration problem here as well right we basically want to learn just enough about each of these pools to execute our trade optimally so if I'm never executing sort of orders of more than a hundred thousand shares I don't need to know what the probability of a 10 million share order getting executed I sort of only need to know it up to a certain prefix so there's a very simple and appealing algorithm that kind of has a bit of an e/m flavor to it which is you basically just start you know you just start trading let's say at the beginning distributing your orders uniformly over all of the different dark pools then you start getting information about them and you can for each one of them form the kaplan-meier estimator of that the distribution of liquidity behind outdoor pool it's wrong of course okay but then you could just pretend that it's right and do optimal allocation with respect to those estimated distributions when you do that of course you will then get new information based on the orders that you submit it and then you can just rest emits a simple you know do greedy allocation under your current distributional estimates then r es tomates with the results of the next round of executions and one can prove that this converges quickly you know with a polynomial rate of convergence to the optimal allocations that you would have made if you knew the a distributions from the start and you know it if for those of you that know a little bit of the reinforcement learning literature the analysis of this is actually quite reminiscent of the analysis of things like e e cubed and r max which are sort of provably convergent algorithms for learning and markov decision processes here we don't have a Markov decision process but the state is sort of our informational state about the different liquidities of the different exchanges rather than an actual okay so I had another vignette that I won't talk about but you can go look on the slides at the pointer to the paper at lease and let me stop there and take any questions if there's time or we can just morph into lunch connection between your paper and Eric's and a way to put it together to make it relevant for ultimately so obviously you're not the only one doing this on the street and there but there are a finite number a relatively small number of hft adoptive doing this so that means that you can actually run a very simple simulation where suppose that you're not the only one using your algorithm what if there are another two or three other players are using similar algorithms but perhaps have different slightly different parts of the data can you figure out what that interaction might produce in terms of the outcome of prices not mathematically but computationally and then to connect it to Eric's paper you can ask ask the question what different market structures with different kinds of algorithms are going to be ideally suited for market stability versus fragmentation all the issues that regulators care about and then start asking more sophisticated questions about how markets structure and algorithms interact so it almost seem like you can combine what you're doing with what Eric is doing in a way that could be really interesting yeah I mean I mean certainly one could do simulations the problem is you know kind of saying what it means to be doing something like what we're doing right I mean I I kind of this isn't quite an answer to your question but I but I sort of am of the belief that you know things like the flash crash or even like the Oh eight crash you know the broader crashing oh eight I mean there's a sense in which the more automation we have and the more data that's available the more inevitable it is that we're going to have incidents like that simply because when you have many many parties that are all using something like machine learning on the same historical data that's going to correlate our behavior automatically like without any anybody directly intentionally trying to copy anybody else like just take this execution problem I described if any of you thought about this problem you would have come to kind of a reinforcement learning approach if you then said well let's go optimize that you would have ended up with you know things that were probably numerically very similar to what we ended up with in our policies okay so you know in some sense right I mean people talk about like the smarts on Wall Street in the quants and and you know but but at the end of the day right the first-order effect here is that our behavior is being correlated through shared information and our all kind of optimizing with respect to that shared information and this was just not possible thirty years ago right the data wasn't there I mean we could imagine and I think that that just forced there to be greater diversity in in kind of the approaches people took to the same problem that's kind of being you know that sport in and so I mean the reason I relate this to the Oh a crash is that you know the sort of you know maybe not the source of the crash but the final steps which was this liquidity crunch you know like lo and behold all these hedge funds suddenly find when they tried to go start unwinding their portfolios that we all have the same portfolios so you know we're all short Johnson & Johnson and so we're all trying to buy back those shares to cover our short positions at the same time and we all probably got to those portfolios through this kind of group optimization so if you put together these kind of algorithms and potential correlation that actually has some very significant implications for whether you should use continuous versus batch options which gets back to Eric's paper so I think that this could actually be a really interesting combination of demonstrating for very job I agree I think there's great research opportunities though though you can see I'm coming from this other side of the fence which you know is kind of deeply mired in the details of the way the exchange is currently work and so I think the hard thing in this to sort of formulate research questions here is sort of what details can you safely throw out and remain relevant to actual practice and which are essential and I don't you know it's a hard hard question in general but very hard in financial markets yeah so I you are you ever going to take the signature of another hitting from like oh um we haven't we have we don't spend time really looking at that because we're not we're actually not engaged in hft per se we we we need to deal with hfts as counterparties I'm sure we're trading with them all of the time every day I mean you you anybody who's buying or selling stocks at some point is indirectly you know trading with them as a counterparty because they're a very large fraction of the trading volume but it is definitely possible to do this I mean you can definitely after the fact like tomorrow you will see what this must indicate executed by that yeah I mean it would it would be a statistical knowledge right you wouldn't actually say you wouldn't be able to go and say like well you know look at this sequence of I mean because people are breaking up their orders over time and and and and venues right so you wouldn't be able to say okay you know I'm going to mark in red each one of the trades there was a child order of some large block trade that somebody was trying to do if you definitely be able to say it's extreme you know just given the statistics historically and in this period it's quite likely that there was you know some large party trying to execute a high-volume trade in a short period of time those kinds of signatures are definitely in in the in the data and you don't need kind of super super I mean it would it would be available even from just like the execution date like trading quote data you wouldn't need to do like limit order book reconstruction to see those kinds of signatures it would let people go to lunch if we're going to have any chances starting on time thanks

Channel: Simons Institute

Views: 72,537

Rating: 4.9373164 out of 5

Keywords: Simons Institute, UC Berkeley, computer science, theory of computing, Economics and Computation, Michael Kearns

Id: XNZ7o3621FY

Length: 54min 49sec (3289 seconds)

Published: Fri Nov 20 2015