5 Forex Scams Used in EAs, Indicators and Signals - Fabricated Backtest Results Scam
Hello again, my name is Rimantas Petrauskas, and in this series of articles, I present my course on how to avoid Forex scams. I show you what they look like and how they work, so you can have a better understanding and, possibly, avoid falling into traps and losing money. Today's topic is another common forex scam - fabricated backtest results.
I will begin by explaining what an Expert Advisor backtest result is.
When you code a Forex trading strategy into an algorithm, you are generating what is called an “Expert Advisor.” Essentially, you are creating a robot-like app for MetaTrader4 that will trade your strategy nonstop; it will monitor the markets and trade on your behalf. All you have to do is leave your computer on and it will keep working.
If you tested the performance of that algorithm on price data from 2003 until today, for example, you would be doing what is called a “backtest.” A backtest uses historical price data to simulate what would have happened if you had been trading that strategy since 2003. In other words, you would be applying the trading strategy to the past to see how well it would have performed.
Obviously, this does not mean that the strategy will continue to perform the same way in the future; any backtest should be considered only an example, much like a viability study would be for any business. Let’s say you were thinking of opening a flower shop. “Would it be profitable?” you wonder. You would look at the marketplace and see that, in big cities, flowers are sold on every block. You would assume that florist shops are viable businesses. If you looked at the past, you would also see that this type of business has worked for decades, even centuries. Since there is no flower shop in your neighborhood, you open one. As long as there are people in your area, there should be a market for your flowers. By going through this process, you conclude that a flower shop is a proven business model.
Backtesting a trading strategy is no different; it’s like conducting a viability study based on historical data. It’s important to emphasize, however, that when you are considering a specific strategy, you can’t look at the last 5 trades, or at the 5 days or 2 months of trades, and decide, “I will give it a try.” Without proven, long-term results that the strategy works, you would be probably wasting your time and money. If you have backtest results for a sufficiently long period, you can better assess whether the strategy might perform well in the future.
As with anything, not all backtests are accurate or true. Some people don’t know how to run the test correctly. Some know the weaknesses of backtesting engines and do it wrong on purpose to trick people into thinking that the strategy performs well.
Fortunately, these days, we have access to Myfxbook, where we can see real trading results and evaluate strategies. Before Myfxbook, a backtest was the only way to know whether a particular strategy had potential. People would present MetaTrader4 statement reports but they are as easily fabricated as backtests. With that said, not every backtest is intentionally fabricated to fool people. As already mentioned, some may have been conducted improperly; they may not necessarily be scams.
In the picture below, you can see two backtest results. The one at the top has a nice equity curve that keeps going up, while the one on the bottom keeps going down. They show contradictory results even though they are backtests of the same EA strategy, tested during the same currency period, with the same modeling quality and timeframe. All parameters are the same; the only difference is that one was performed on MetaQuotes Data that, by default, reaches only 90% modeling quality and forces you to use a fixed spread, which – in a real trading environment – is not correct because, nowadays, most brokers have a variable spread that changes every second.
The backtest at the bottom of the image shows result of 99% quality. It uses EveryTick data, which includes the option of using a real variable spread. Keep in mind that this is not something you can do on MetaTrader4 alone; you would need additional software for that. I use Tick Data Suite, but there are other options. However, only Tick Data Suite uses real variable spread action. I want to make it clear that I am not promoting Tick Data Suite; I am only saying that it offers the option of running a 99% backtest and this makes all the difference because it can prove that a strategy that looked good at first glance is worthless, as in the above example.
Please continue reading or watch Fabricated Backtest Results Forex Scam Video #5
What you need to know about backtest results
- A backtest allows you to see how a particular strategy (EA) would have been performed in the past. It’s no different than the flower shop example. You want to see whether this type of business has worked in the past, rather than venturing into something completely new and untested. That said, I am very open-minded and always in favor of trying new things; at the moment, I’m testing approximately 2,000 strategies in my demo and real trading accounts. However, if you want to trade for a living using some trading EA, you should not go into it blindly. You should know what’s behind the algorithm, what strategy it uses, and understand some of its logic. See its backtest. If it has worked in the past, there’s a good chance it would work in the future.
- A backtest is just one piece of the puzzle in algorithmic trading. A backtest gives us a reason to continue with strategy EA development or testing, as well as confidence that we’re not wasting our time. If a strategy does not work on the history price data, dismiss it, and find something else. If you see good backtest results, however, it has potential. You can work with it, try to optimize it, test it on a demo account or small accounts, and so on. You can put some time and money into it. Maybe it will turn out to be a great EA after all.
- A backtest does not guarantee future performance regardless of its accuracy. Even if a 99% Every Tick modeling backtest looks awesome, it doesn’t mean it will perform that way in the future. It only means that there’s a chance it might. There are no guarantees in Forex.
- Never judge the trading strategy by its backtest alone. You need to run additional checks called “robustness tests.” Basically, you run an additional one thousand backtests using different parameters, maybe skipping trades or shuffling trade order, or changing spread for each simulation, in order to see how the strategy behaves in different conditions. In other words, you try to break a strategy in every possible way. If it breaks, you dismiss it. If the strategy survives all your tests and you cannot break it by changing the parameters, then it has a lot of potential. I would apply that strategy to a real trading account.
- Always check whether the backtest was performed correctly and accurately. Not many people know how to perform high-precision backtests. It’s important to understand this because there are many ways to do a backtest. There are many small things to look at and adjust, so it’s very easy to miss something in the configuration. If you miss something, it means that your backtest was done improperly and that your results are inaccurate.
How to know if a backtest was done correctly
Now I will explain how you can determine whether a backtest was done correctly or it’s a scam. This knowledge will come in handy if you’re considering using a strategy that someone sent you or that you found online. Or, perhaps, you ran the backtest yourself and want to verify that you’ve done it correctly. Obviously, I cannot explain how to do a backtest here, but you can refer to this online tutorial on MT4 backtesting. All I can do in this article is to mention a few things that you should check:
- Check the backtest type and its modeling quality. It should have been performed using Every Tick with a 99% modeling quality. I would consider such a backtest of high quality and accuracy. If it shows good performance, I might continue with EA development and testing. In the screenshot below, you can see one of my backtests reports with a modelling quality of 99%, and a graph showing equity and balance. It keeps moving up, so it’s a good strategy. You can also see that it shows 99% in Every Tick mode. Modeling quality and Every Tick mode should always match. If they don’t, someone is trying to trick you. So, the first thing you should check is modeling type and quality.
- Check what spread value was used. There is no way to know exactly what spread was used during the test unless it was a fixed spread – you can see this in the backtest report. In the picture above, you can see that the spread was set at 30 points (3.0 PIPS). But because I was using variable spread, this doesn’t count. It’s just a number on the report, but it’s not real. If this were a 90% backtest – not 99%, but 90% or some other modeling type – then the spread number would indicate what spread and points were used. If it’s 99%, it means the spread was variable, so the backtest is quite accurate. If the modeling quality is anything other than 99%, it means that the spread was fixed and the backtest is not accurate because brokers don’t use fixed spreads. Very few brokers still have fixed spread and only on some account types; most use a variable spread these days.
- Check the mismatch errors. Always check how many mismatch errors the report has – that number should be zero. Here’s an example of a backtest report with 1,761 mismatch errors, which is a huge number. This indicates that the strategy report is not accurate. You should try to fix this.
- Check the period of the backtest and the number of trades. The period should be around 10 years otherwise it should have at least 100 or, better yet, 200 trades. Actually, the number of trades is more important because of the statistical significance. Here’s an example:
In the above screenshot, you can see that this backtest report shows that the data installed on the testing environment runs from 2003 to 2015, but the backtest itself is conducted only from 2010 to 2015. In other words, the timeframe between 2003 and 2009 wasn’t used in this backtest. This could mean that the creators of this backtest are hiding something – they do not want you to see those seven years of results. In this particular case, however, we can see more than 1,000 trades and this carries great statistical significance. The large number of trades indicates that this is a good backtesting report. We have a good number of winning positions. It has 99% modeling quality, a lot of trades, Every Tick, and so on. It’s a legitimate backtest that looks quite good. If there were only 50 trades, or something less than 100, it wouldn’t mean much.
- Check whether the modeling type and quality of the backtest match the graph. I mentioned this already, but here’s an example of a report with a 99% modeling quality while the graph shows only 90% modeling quality. Since those two do not match, this is a fake backtest result.
Now, I will show you how easy it is to change the modeling quality – basically, this is how scammers trick you into believing that their backtests look good. At the top left corner of the image below, you can see a field highlighted in blue.
All I have to do is type in that field that the content is “editable,” turn “design mode” on, and press “enter.” This allows me to change anything – the modeling quality, the strategy name, the timeframe, even the trades although that would take time. What I’m trying to say is that backtests are very easy to edit. When I have finished editing, I turn off “design mode,” and that’s it; I can grab a screenshot and send it to someone saying, “Hey, look at this awesome backtest.” So, if you see just a backtest, don’t trust that it’s accurate or true; it could have been fabricated very easily.
I hope that you’ve found this article helpful and that you’ll join me for the next one where I look at Fake MyFxbook Accounts ScamEditor's Note: The original video on this page was produced by Rimantas Petrauskas. The text version was prepared by the FPA. The text follows the primary concepts in the video, but has a number of differences in wording.
Probably not :-)
Always very useful and easy to get.
Always very useful and easy to get.
What you explained is modifying the bactest results (report) through HTML.
But, can it be manipulated during the backtest proses itself? Hence it generated a good result without the need to edit the results (report).
I once read people argueing in MQL5 forum about an EA for sale, which was not perform well during real trade. They compared real trade for several months and then run backtest on that same months. And the result is different.
I am curious about the possibility that EA creator can manipulated the backtest trades rather than backtest result (report).