This section describes some of the details you may need to know in order to produce Metatrader exports effectively. If you are familiar with the way in which MT4 works, you can go straight to the section on MT4 Backtesting Quick Start Guide that will get you performing 99% back-tests quickly.
MT4 has the ability to write an test custom trading strategies in the form of computer programs known as “Expert Advisors” (EAs). EAs essentially allow the specification of a computer program that allows it to place trades with your broker - in a totally autonomous fashion, if need be. EAs are often sold via various platforms such as Metatrader's online store. EAs can also be written by users to test their own strategies. EA code is written in a language called MQL which will look familiar to programmers who know the 'C' Programming Language. An example of this code is provided in the standard MT4 installation called “Moving Average EA”. This EA demonstrates a very basic strategy that opens and closes trades when a moving average cross-over occurs.
Of course, having provided the ability to create EAs, Metatrader also provides the ability to test them to see how they perform through a process called “back-testing”. Back-testing is simply running an EA through a chronological sequence of historical data to see how the EA would have traded - and ultimately performed - in past market conditions. The Metatrader Strategy Tester is this back-testing tool and it is already installed as part of the standard Metatrader package.
You can show the Strategy Tester by going to Metatrader's View→Strategy tester option. A screen similar to the one shown below will appear.
It is out of scope of this document to provide a full explanation of the functionality of the Strategy Tester. There is a breadth of information in Metatrader's own Help manual and also online forums to assist you further.
In normal operation, the Strategy Tester sources its historical data from your broker. Users have often complained that this broker data is often full of gaps and not complete for the purposes of the back-test. Not to mention one of the major draw-backs of this data is that the most granular information that is contained is 1-minute bars. This means that from a composition of the many ticks that typically occur in a single minute, the historical data only passes through the Open, High, Low and Close of the 1-minute bar. As you can imagine, this is unlikely to yield an accurate assessment of an EA that trades often or is dependent on small price movements (as many EAs are).
Enter Tickstory. Tickstory allows you to generate a tick-by-tick historical database that is used by the Strategy Tester. This allows you to back-test your EA with more granularity and this edge can enhance your assessment of your strategy. Not only this, but Tickstory also generates complete charts from this data which means that when you display your bar charts within Metatrader, they will be much more reliable and complete.
MT4 stores historical data in its own custom format which have the extension *.HST and *.FXT. These files are used by MT4 to display information on charts and is also used in the process of back-testing, respectively. The historical data files contain important information for MT4 to evaluate a back-test such as: oAccount details such as the leverage on the account, commissions, etc. oInstrument details such as the lot size, number of digits, tick value and so forth. oPrice information, including bid price & the bid/ask spread is for a particular instrument.
All these details are embedded within the historical database which means when your EA requests information (such as the instrument lot size), it goes to this database to retrieve the details. When MT4 builds its historical database, it sources all this information directly from the broker, but effectively only stores at most the 1-minute bar data of the ticks that are coming in for your broker. When you perform a back-test, information stored in the database history (*.HST files) are used to produce an FXT data file for the purposes of back-testing. FXT files contain the detailed “tick-by-tick” information that is used by the back-tester. An FXT file is generated for each of the time-frames for which MT4 permits back-testing in - 1 minute, 5-minute, 15-minute, 30-minute, hourly, four hourly, daily and weekly. The screen shot below shows the various back-testing periods that are supported by MT4.
The Tickstory system allows you to produce a historical database from pure tick data and replace the existing broker MT4 data with this much higher-quality information.
In order to utilise your own custom exports, it is handy (but not obligatory) to know where MT4's historical database resides. You can locate your HST and FXT files by going to File→Open Data Folder option in Metatrader as shown in the screenshot below.
Selecting this option brings up a Windows Explorer window to the location of where all your Metatrader data files are located. This folder is typically located in the C:\Users\<YourName>\AppData\Roaming\MetaQuotes\Termimal\<ID> folder. If you are using Metatrader's “portable” mode, then the MT4 Data Folder will exist in the same folder as your Metatrader installation (eg. C:\Program Files (x86)\Metatrader).
If you browse to the history/<broker server name> folder, you will see a series of HST files pertaining to each instrument and timeframe. The “server” directory that contains the actual historical data (*.HST files) that were downloaded from a specific server. Often brokers will have two servers - live and demo - and the data could be different in each database. For the purposes of back-testing, it is recommended that you are connected to the 'Demo' server.
If you browse to the tester/history folder, you will locate all the FXT files:
Note how there is an FXT file for every symbol, per time period. For example, EURUSD has a file for 1-minute (EURUSD1_0.fxt) all the way to daily (EURUSD10080_0.fxt - where 10080 minutes = 1 week), weekly & monthly bars. Depending on the time-frame you back-test in, this particular FXT file will be used to evaluate your back-test and contain all the data that is required including pricing data, spreads, valid instrument lot-sizes and so forth.