Algorithmic Trading via ZeroMQ: Python to MetaTrader (Trade Execution, Reporting & Management)

this video follows on from the first webinar we conducted on Python 0 mq and Metatrader the integration we discussed therein it went through everything from the beginning in terms of y 0 and Q what it is the integration possibilities between 0 and Q and Metatrader and how I thought fit into the picture here we're continuing on from that conversation in the context of trade execution management and reporting using Python in terms of prerequisites it would be useful if viewers of this video have some programming experience that could be in one of the following one or more of the following programming languages these being compliant all these being compliant of course with zero and Q as in 0 m q has bindings for these programming languages C C sharp C++ Python R and mql of course that's required for programming the Metatrader side of things that we're going to discuss shortly trading experience with medicator 4 & 5 not necessarily both would be really good since you understand how things work in meditator and how python i can then serve those features within your Python environment allow you to do the same things that you would do inside of Metatrader knowledge of mq l EA functions so this tutorial is optional since we've done a lot of the work in terms of developing functions the methodology and the functionality within the mq l EA that's provided open source on our github profile and we'll discuss elements of that shortly describing code structure everything from functionality and how python and 0 mq python buyers your cube talks to Metatrader it must be stressed here though that python isn't necessarily the only language that this functionality can be developed you are absolutely free to use any programming language that has a compliant binding available in add 0 Q something we discussed in the webinar where we talk about how to interface your queue how to interface Python with Metatrader bios you some meta critter terminology is good to know here for example lot sizing what our magic numbers ticket numbers order types or piece ll be bye-bye stops cell stops etc that are available to you inside of Metatrader in terms of the agenda will discuss one additional 0nq messaging pattern the publish/subscribe messaging pattern that wasn't included in the first segment when we talked about how to interface python r with military adviser you initially you were using the push-pull pattern and the request reply pattern if you it would be said it would be beneficial if you could watch the replay of that webinar first how to interface python r with metadata for that is available on the darwin x youtube channel and then visit this video for good context and to follow through smoothly we'll discuss what's new in this implementation so we've essentially rewritten everything from scratch to serve a few purposes firstly the code has been revised so it is now pipe in Python 3 as opposed to Python 2.7 that it was it was essentially in the beginning metatrader4 side we have a completely revised expert advisor in the first webinar the expert advisor achieved minimal functionalities minimal functionality such is to demonstrate how things actually work on how you can build upon it we received a lot of feedback regarding the same and we decided to ends actually code up the entire thing and give you everything in terms of functionality from opening modifying closing orders requesting account summaries open positions magic numbers etc etc all communicated between MIT and Python and Metatrader in a standardized format and jason in this case so we've done a few upgrades to it i'll describe shortly essentially at this point an algorithmic trader using Python or any other 0 mq compliant programming language can use the expert advisor as is however caution is advised at all times please make absolutely sure that everything in the code available to you on github is per your specification is per your individual requirements there are if there's anything missing in terms of code you are highly encouraged to first read all the code and then move on from there especially if you plan on using it in a live trading environment we'll talk about trade execution and management basically following on for myself from what I said earlier we'll talk about trade execution and management account in trade reporting and then we'll talk a bit about what comes up in the segment after this which will be dedicated to subscribing to market data between Python and metatrader4 via 0 mq right so let's begin now this segment would be slightly different from the first webinar where everything was in a PowerPoint we're actually now going to go into the environments themselves that's metatrader4 my Python environment which I use spider part of the anaconda project and of course we'll use benefit so let's get out of presentation mode and go through to the environment so in terms of explaining code will start with Python the script has been completely revised the script is now called firstly DW x 0 mq connector it is available on our github page let me quickly go there to show you where that is and it's on Darwin X labs so we'll go over to Darwin X labs if I can find it let's go ahead do that there we go right once you get the Darwin X labs the github repository you head over to tools and the new code both Python and MQL components of the code are in this directory DW x 0 mq connector click on that and you're taken through to the main repo a main sort of brown chair and the first two lines are important in the sense that the second line really is important this is where the latest version of any yeah the latest version will always be present here so this line may change in the future and indicate what version we're running on at the moment you simply need to just visit this page and click here as it says to go through to the current version and add this at the time of recording this video the current version is 2.0 point 1 the release candidate 8 on this page you'll also find a lot more detailed information as opposed to before where we just had the scripts up so we've done a bit of work here to make things as fluid as possible and as detailed as possible for you to hit the ground running but introduction which is when you have some time over a coffee you can read through it discusses what 0q is why we need it what client-server model is that we're going through here and building upon it talks about it goes on to talk about installation the project's met the requirements in terms of libraries and environments you'll need Python of course when it says minimum version 3.6 that is really to say that we developed this word with a minimum of 3.6 we haven't at this point in time release any information about backwards compatibility the code is definitely written for Python 3 but with a little bit of modification and of course be ported on to Python 2.7 if that is what you are using and for some reason don't wish to trade the waters of Python 3 you'll need Lib zmq the minimum version that this implementation was tested on those four point two point five this may change in the future of course with future versions so it's really no point in me going through all the libraries and their versions the library's main libraries you need are the first two and then by zmq Lib sodium open source available on github and peel for lip open source available on github and also an ELC mq the instructions are all provided here I won't go through them in this tutorial if you have any questions you are of course free start an issue if it's a problematic question or simply email us at info at Darwin Dexcom and we'll get back to you with additional help as regards these instructions so in terms of configuration I'd like to spend a few moments here the EA the expert advisor on the mql side as I mentioned earlier has been completely revised and includes a lot more functionality than before and hence input parameters so the first input the first few important input parameters are of course the 0nq protocol that we'll be using which is tcp hostname Asterix refers to your local host machine if you are connecting to a remote machine you'll need to specify an IP address over here as well as make sure that the permissions and firewall configuration on the other side allow you to connect to that machine the push port represents the the socket the push socket that will create the pull port is for the pull socket will create and the pub port this is the new request this is the new 0 mq messaging pattern that we'll talk about in just a second this is the port for that publisher socket millisecond timer you'll figure this out if you simply go into the event timer code that I'll show you in a second inside the military expert advisor this will show you where mili second timers come into play and of course the default magic number maximum number of orders are one so you can of course change this to to suit your purposes with the default value is one maximum lot size is set at the moment again for safety reasons to the minimum which is 0.01 you can of course change this to whatever lot size you would like to execute now one note on the lot size this script assumes that the lot size is going to be equal to or less than this maximum lot size so whenever you send traits from Python or any other programming language where you're sending rates to this expert advisor you need to program things such that they respect this maximum lot size otherwise of course the trade won't be executed maximum slip HTML is not a direct man accessmode is true and this last configuration parameters you know you can regression publish market data is by default set to false the reason for this is that we'll talk about market data pricing retrieving data both historic historical data as well as real time build a skater from Metatrader 4 or 5 in the next segment of this 0 mq series of webinars and videos so today we'll set this to false and not tell them into that run until the next video here's just what the journal output looks like experts are portrayed as publishing data and we'll go through each of these code segments here in a second and also demonstrate how things work in a live environment will use my Python environment I've got Metatrader open and connected to an account which is a demo account and the zero and you DW DW x 0 mq server expert advisor as per the instructions you would have followed by installing and configuring everything is also successfully loaded on the chart and functioning let me take a look at my properties to show you have these look so they're exactly the same what I'm doing for the moment just to demonstrate to you how the retrieving market data bit works very quickly is I've set it to true but in the next segment of this series we'll go into a lot more detail on how this works and storing and manipulating data to our for our purposes so just going to cancel this now and the expert advisor is loaded so moving on to the first part of the exercise sending a trading command from Python at a meditator for the very first thing we want to do is I want to restart my journal so that I can show you from scratch everything I'm doing colonel will restart in a moment and then we're going to load the script for execution right there we go so what you'll notice is a class called D to D WX 0 mq connector inside of which there are several functions available to you the functions we'll be covering today will be trade execution specific functions and I'll show you here how to use them I will let you peruse the code yourself and of course there is github with a handy-dandy description of every single function available to you and this script a note on the the way to send orders so in the Python script you'll notice that we have a dictionary object that encapsulate sin firm ation that you'd like this into a meditator for it has to be sent in a format that the expert advisor understands processes parses execute and then sends back a response in JSON format the default order dict function generate default or reject function creates a default dictionary object for you that contains open specifying open a new trade type zero here you is the price zero point zero etc et-cetera et-cetera a note on top losses and take profits these are specified in points so when you're developing or incorporating this code in an existing python or any other trading strategy then you have to account for the fact that meta trader is expecting you to send stop losses and take profits in terms of points and not pips the order type corresponds to meta trader order types which are opie by or piece l of VI stops I'll start by limit set limits so this integer value corresponds to the integer value of those order types and we'll quickly go over there and find those order types so I can show you what these values correspond to types of orders trading principles set it by limit etc etc etcetera I think I haven't found the numbers so finally let's let's use an example so OPP I and Q out that will order order type here we go these are the older type integer values that meta trader is expecting and these are the only ones that you can hence use from the script moving on to actual implementation so we're going to create ourselves an object I'll call it ZM q and create an instance of our class like so the first message is you get R then it's been initialized and it's ready to send commands to Metatrader viable push port three to seven six eight and it's listening for responses from meditative via the pull socket three to seven six nine inside Metatrader everything is loaded as is and it's essentially waiting for us to execute basically communicate when they tell it's something to do so go back to Python and the very first thing we'll do is retrieve that generate order dictionary function that we talked about just now and see what we're going to be sending meditated by default so we have we're asking Metatrader to open a bye-bye trade on the Eurodollar specifying a price of zero tells meditator to open on market order with a stop loss of five hundred points which are 50 pips on the Eurodollar take profit of five hundred points which is also 50 pips on the Eurodollar a comment we'd like for meditator to include in the trade the lot size that we'd like now remember this is zero point zero one in this default order dictionary and you know that in the military expert advisor settings we've also set our max lot size to 0.01 so Metatrader will not execute a trade if you send it a lot size of greater than 0.01 surfer for sake of demonstration will stick to 0.01 one micro a lot and followed by the magic number to just set a random number one two three four five six and ticket number to zero because ticket numbers are generated by Metatrader and sent back to you so we can't generate a ticket number we can only use an existing ticket number to perform operations on the trade that it corresponds to so the first thing we'll do here is underscore ZM q dot I would like to send a trade if you type DW X and if you're an eye Python console you'll see all the available functions that start with those characters in this case I'd like to execute a new trade so firstly we will just see if any orders are open let's have a look at that to do that you execute the dwx MTX get all open trades function and you'll see you'll see that it returns an empty dictionary because in Metatrader right now we have no trades in execution let's work backwards and test that functionality a bit more I'm going to randomly execute it this is a demo account so I'm just randomly buying something on the Eurodollar here for a 0.01 Watts now I have a trade open in that trade has gone into execution so I can go into my Python environment enter that same function again and now Metatrader tells me that python tells me that Metatrader is sent back a response in JSON format with open trades being this only trade here with lots is 0.01 stop-loss of 0 TP of 0 and its current P&L if I want to see what the P&L is after a little while all I simply need to do is keep executing this trade to get the latest information from Metatrader so it's in real time it will send the instruction to Metatrader and always get you the active actual response now now that we have access to the ticket number say I wanted to modify the stop loss and take profit values when I click buy on this it said no stores no take profit so what I need to do here is I need to send an order I need to send an instruction to Metatrader that could you please modify the stop losses and take profits on this position so on this trade so what we'll do is we'll tell zmq underscores m q dot underscore TW what i would like to do is modify a trade by ticket and you'll see that the ticket number is requested as well as the stop loss and take profit so here I'll enter this ticket number that I see in the MTX get all open trades which is eight six zero two one one four nine so eight six zero two one one four nine of course in your logic in your actual trading all of this can be automated so you would develop something that reads this inbound dictionary inbound JSON response from Metatrader and processes it processes it and does all of these things accordingly the sake of demonstration Here I am typing these things in manually which is very very inefficient but for sake of demonstration it serves its purpose so we're going to set the stop loss to let's say I want to set the stop loss to 20 pips now remember Metatrader is expecting you to specify the stop loss and take profit in terms of points so we'll enter 200 points which corresponds to 20 pips on the Eurodollar just remember the symbol return is your dollars the euro dollar trade is in execution and I'd like three to one risk reward reward risk ratio here so I'll say stop losses 60 pips which is 600 points and that's it so I'll send this instruction by hitting enter and Metatrader sends back a response saying that yep modify was executed and 20 pips was set as the stop loss and 60 pips was set as they take profit Metatrader on the expert advisor side has calculated the price levels required for this and inserted them into the existing trade so now you see that the open the open price of this trader is 1 point 1 3 5 1 3 the new stop-loss is 1 point 1 3 3 1 3 which is 20 pips away from the open and then you take profit is 1 point 1 4 1 1 3 which is 60 pips above the open price on this buy trail and so now we've been able to execute an order modification a trade modification say I'm for some reason I have made the wrong decision or for any other reason I want to close this position again all I need to do is quickly have a look at what my trades are just to see what's open and we we see that the same ticket is open so we'll go through two underscores @mq why I just did this because you may be executing other expert advisers on the account you may be trading manually there are several reasons for why you'd want to get all open trades again in your Python environment just to make sure that the trades you want to close are indeed the ones you want to close otherwise by sending come like this you must be absolutely sure that you're doing the right thing hence the caution right so I'd like to close this order I can't close it partially because the minimum lot size is 0.01 there's no partial flows available here so what I'll do is I'll close the tray I have a few options I can tell Metatrader to close all the trades this is dangerous if you are using more than one form of trading on your meta trader terminal so form meaning that you could be trading are you trading manually as well as with the 0qa that we've provided do you have other EAS on the account do you have anyone else trading on the account are you yourself or your two-year-old when you're not looking trading on the account there are several things that could be happening so close all trades is the nuclear option where you tell Metatrader to close all trades literally so we only use that one we're absolutely sure that we want to close all orders on the terminal that are definitely we definitely know about but in other cases if you want to play it slightly safer you can either close all trades by a magic number to say you wanted to close the trade that corresponds to the magic number of the expert advisor here which is one two three whoops oh that was a bad move not the right thing to do lie on a video huh okay so we'll just go back to where we were and now have a look at the properties so the properties here specify 0.01 as the maximum lot size and magic number of one two three four five six as I mentioned earlier so my other options here are close all trades the nuclear option close straits by magic number in close trade close trades by ticket since in this case I want to close a trade by ticket I go down to that function completed and into the ticket number that I'd like to close which in my case is eight six zero two one one four nine and that will send the order to meta trader meta trader will return a response a successful response will return the actual function the the the acts performed in this clay and in this case it was yes I have closed the trade of order ticket eight six zero two one one four nine the close price I executed the instruction at was one point one three five or two and I closed this many Lots at market and the operation was a success if you don't believe what Metatrader has sent to you you can of course go to Metatrader and see that the trade is now gone and in the history that was the last trade that we just closed via Python you'll notice here that the closing price is one point one three five zero one but inside Metatrader the closed prior inside python the price that was returned to us was one point one three five zero two this is not necessarily trade this is not necessarily slippage this is Metatrader returning to us the price at which it attempted the instruction and Metatrader once you go inside the terminal you'll see that it's telling us the price that the trade was actually closed that this latent this slippage visibly may be due to the latency between the time you sent the instruction in the time it was executed it may be an actual slippage maybe other things so what you need to essentially do going forward just to monitor these things and see where your slippage or latency factors come into play right so we've done a few things now we've opened a trade we've checked our account for trades we want to know what's in execution we wanted to modify a trade stop loss and take profit we get that we also wanted to close a trade by ticket we did that we haven't practiced some more trade execution functionality here so let's go through what we have available so a right we've got closed all trades closed partial buy ticket closed trades by magic closed trades buy ticket so let's say we wanted to go the nuclear route for some reason we want to close everything on the account for this to happen we first need that many trades on the account so let's go ahead and execute some trades let's send a new trade and by default it will send the default order dict when order is set to none the this function defaults to the generate default order tick function which will open a buy trade on the euro dollar where the stop-loss five hundred pips have five hundred points and I take profit of five hundred points at a price of zero zero which indicates market order and type zero is OB underscore buy which is a buy order so we'll do that a few times that's an execution so let's go to meditate and see make sure we've got everything set up correctly so go to trade and you see that a new trade has indeed gone in and it has a stop loss of 50 pips which is five hundred points and a take profit of 50 pips which is also five hundred points now I want to do a few more of these so I can execute my nuclear option later so we'll go ahead and do that go one more let's do five in total so we've got five zero point zero one lot size trades in execution and Metatrader will have executed all of them the comments will reveal the dwx python to empty comment that the default function the default or detect also asks meditator to specify now we've got five orders all five orders have the same magic number right because it's the one EF on the account so we have two options to close all of this we can say that M Q dot underscore t WX close all trades the nuclear nuclear option close trades by a magic the semi nuclear options which uses the magic number of the EI in question to do that so we'll do here is record closed trades by magic number meta trader in its response returns to us the magic number in this JSON output as well so we'll use the magic number one two three four five six and say go Metatrader will go away start closing those trades and then return you a larger JSON object which will contain responses to each of the instructions you sent it so when you send the instruction close all trades to meditate a meditator with them cycle through all the trades and start closing them one by one once all those trades have been closed it will send back to you the status of each closure whether it was the success or not and the response so in this case all five trades that were open have been closed successfully and the response value is there for success so the idea here is to use the expert advisor in meaningful ways incorporating this code into your existing trading strategy if you're if you have a trading strategy in Python you can of course the easy route here is to simply incorporate the logic that's been written for you inside this new Python script twx 0 mq connector version 2.0 one release candidate eight which will change in future as the versions keep increasing and there are parts of the code that you can recycle so you don't necessarily have to use the functions that have been written for you there they serve the purpose of demonstrating how this communication happens this interchange between Python and end Metatrader works the Metatrader expert advisor will communicate in exactly the same way with any other programming language in which you've developed this functionality so you don't have to worry about python being the the it of everything no 0 mq has several compatible bindings all you need to figure out is which binding supports your program which if there is a binding available for your programming language looking at the code really quickly let's let's go through the code skim through the parts that mean something to us for instance I talked a lot about pull and push and pull sockets and publish reports and things like that earlier on so these are initialized in the on init function the publisher socket is bound to the published report and this one we're not going to go through too much detail in this video but there the next segment will cover everything about market subscription let's quickly have a let's let's have a quick browse of how that actually works so a component of this that we'll talk about in depth in the next segment of this series is market subscription where any symbol that is available inside a material for data for that symbol you can access through your Python environment here through this Python script by supplying the symbol and actually subscribing to the data in real time so there are two things we can do here we can subscribe to the real-time data we can collect historical data that's already in the history Center in Metatrader 4 or 5 so for this demonstration just to show you really quickly how that actually works I will simply subscribe to the market data and show you how that happens in real time so that you have three functions here two functions here sorry subscribe market data unsubscribe all market data requests and unsubscribe market data so we'll go through so when you send a market data request you're actually asking for historical data but if you'd like to subscribe to real-time data you need to hit the subscribe market data function enter the symbol for which you'd like the real-time they'd ask information don't supply the slash sorry and that's it subscribe to your USD bid-ask updates now when I initialized this class object this class instance here I didn't specify my Robo city to true so here everything is happening in the background and there is a variable inside your class instance your object called market underscore data underscore DB this is a dictionary object the keys of which will tell you which symbols are currently subscribed or were subscribed at some point but have data available in them right now we've only subscribed to one symbol the euro dollar so therefore there's one to one object in there called the eur/usd if I want to see how this is doing if there is any data then I simply look at that particular key and we want euro USD [Music] tells us that this data is in here so far so it keeps updating as the data comes in you notice that the last time stamp is reflective of that so you can keep collecting data in here if you wanted to see verbosity if you wanted to see this we're both as verbose output you'd need to restart your instance with the appropriate variable that's specified in the initializer which is it's not specified in the initializer you have to set the set the actual reversal you can see it in the code later on anyway but that essentially shows you that you can subscribe to any market symbol that is available inside of that is quoting inside metatrader4 at the present time you can of course subscribe to more than one symbol as well so I could subscribe to the euro dollar and the GBP and I'm subscribed to both if I look for the keys now I have two keys in there and if I want to see what what date has been collected so far and I can hit that particular key and you'll see that you have a timestamp and the bid-ask double for each of those data tuples that you have available to you and that's it if you want to unsubscribe from any of them is just as simple you use the TWX MTX unsubscribe all market data requests to get rid of everything unsubscribe from all the symbols you subscribe to or you can unsubscribe from a single simple symbol so in this case for demonstrations sake we're going to go ahead and unsubscribe from one symbol that's the first unsubscription that's gone through and that's the second unsubscription that's gone through so you'll notice now that your data stops at those timestamps unless there's something still in the pipeline and yeah that's the procedure for subscribing to market data again this code demonstrates how to achieve this this market data DBE class variable is set up to store this inbound information to demonstrate how this information looks you don't have to use the class the code in exactly the way it's written know you feel free to incorporate this code or incorporate the logic into your existing code in fact I encourage you to do so because that boosts creativity and if you have anything fun and exciting and more efficient to share please do we'll always always take a look at it we'll always publish it on github we'll always reference you in it give you some exposure as well as feedback as to what you've done and how its benefited you and will benefit others since it's open source if it's open source and on that note we've covered everything so far on in terms of the agenda we've talked about trade execution and management so the first thing we did was we executed a trade we wanted to manage its position because the stop loss and take profit were zero we sent stops and take profits of 20 and 60 pips respectively to Metatrader that was then executed we then closed the trade so those were the elements of trade execution and management that I wanted to cover which is opening modifying and closing trades account and trade reporting was covered in the sense that you have your you always have the ability with this new expert advisor curve as well as the new zero key bridge script to always obtain your open positions in real time so whatever's open in Metatrader you get it all and then you manage your positions accordingly depending on your criteria as I discuss the criteria being do you know every single position and every single trade that's open on your on your terminal if you do then going the nuclear options of close all or close all or close all by magic etc are are safe to use but if you are trading on an account where there are multiple expert advisers where you also might be trading manually you should be particularly careful and use those closure operations by ticket and at most by magic number close all literally close everything on the terminal that the expert adviser this expert adviser is deployed on what's next we discussed demonstrate I'd I demonstrate a little bit of this which is Margaret subscription we'll cover this in a lot more detail next time we'll see multi-threaded a multi-threaded bid-ask retrieval from Metatrader into your Python environment we'll also see how to get historical data efficiently from meta traders history Center that over zero MQ is quite challenging depending on the amount of data that you want so we have a few other options to use there and we'll cover both will cover the 0 mq option we'll also cover the flat file read option both of which will be implemented in Python and shown to you in the next segment if you'd like to read read more about the publish/subscribe publisher subscriber messaging pattern feel free to go to read the docs which is over here let's go to push for learning 0 mq with pi z mq dot read the doc start IO and over here you get a description of all the available 0 mq messaging patterns devices multi-process bias and queue polling and sockets and other information I highly encourage you read these because it gives you a much deeper understanding of how this messaging pattern works and how it's being used in the Python script as well as in the expert advisor and the publish/subscribe one is the new one that is part of this new implementation the Revised Code version 2.0 point one we won't talk about it right now feel free to read up on it and read the docs but in the next segment the next video we'll be talking specifically about market subscription between Python and metatrader4 we'll go into this with a lot more detail and cover other options that we can use as well thank you very much for putting up with it's been exciting to revise everything to Python 3 the revised functionality add a lot of functionality to both Metatrader the penetrator expert advisor as well as the Python script and we'll see you next time in the next video if you have any questions whatsoever please feel free to comment below if this video is on YouTube or anywhere if this video is accompanied by a place where you can post comments otherwise feel free to write to us at info at Terminix comm with any suggestions feedback etc as regards this implementation if you need any help understanding how the code is structured feel free again to write us to the email mentioned earlier or create an issue if it's a problematic situation create an issue on github and we'll deal to it as soon as we can thank you very much for your time and see you next time

Channel: Darwinex

Views: 28,937

Keywords: Forex Trading, FX trading, Investing, zeromq, algorithmic trading, python, metatrader, cfd, darwin exchange, darwinex, darwin

Id: 3nM0c2kG_Sw

Length: 40min 22sec (2422 seconds)

Published: Fri Feb 22 2019