Expert Advisors Fraudulent Backtests

engtrader · (#1 (**permalink**))

It has come to my attention after trying a few commercial expert advisors and finding that they don't really match past performance that some vendors actually have the backtest settings hard coded inside the code and have them change on certain dates (all done internally) to make the backtest look good but on forward testing obviously unlikely to work.

I have been told of 3 that do this (I won't publish here as its 2nd hand info), has anyone come across this and have first hand information as to who does it. This would constitute fraud in my mind and make me nervous about any EA advertised!!

Pharaoh · (#2 (**permalink**))

Hard coding backtest results - that would definitely be a new and different fraud.

It should be possible to prove this. Find a way to seriously corrupt the data set for backtesting (like swapping names between EURUSD data and EURGPB data) and see if the results are identical to those when the data isn't corrupted.

engtrader · (#3 (**permalink**))

Yes its a bit insidious as you wouln't know as the backtest may look ok, that what is scary about it, I guess that's why you should always test on a demo first. I only became wary of this when I bought an EA then tested and noticed:

1) The backtests could not match the data on the website
2) The stop & profit targets were different on different sets of data at different time periods
3) The backtest settings could be changed but made no difference to the backtest results (ie the actual settings were hard coded internally)
4) The vendor promises regular 'updates' ie newly optimised data

Unfortunately only those with access to the underlying code will ever know.

Buyer beware!!!!!

Michael Frisom · (#4 (**permalink**))

Hi Guys:
It is a hot new, The NFA should make something about It, they sometimes spent more time controlling the investors believes thant the real problems in FX market.
Michael Frisom
Department of Statistics
Iowa University

ernest8fingers · (#5 (**permalink**))

Who cares about BACKTESTING anyways? All that matters is forward testing and if its "LIVE" so much the better. Believe nothing! I am working on an EA (for profit) and will offer a free unlimited Demo version for download- if the person likes it-they then buy the version for "Live" trading(and not for a 1000.$ bucks either!) All the hype about the "biggest"-"best"-EA are only marketing tools after all. Offering a free test drive of product is the best way to get word out and prove the worth of software and its company-anything less is suspect.