Intensive Review Genetic Builder Intensive Review

[Mark Fric]

thank you for all the feature requests, I'll put most of them into the new version. Trailing stop is planned, as well as some kind of risk/reward ratio configuration. Next version will have more flexible dismiss conditions, so you'll be able to configure it to dismiss strategy if it has less than X trades.

Quick question - Assuming I'm telling it decimation of 10 and population size of 100, it will work through and create 1000 strategies. What happens if during that process I go into the results and manually kill off some of the less promising strategies? I.E. If I go in when there are 600 strategies that it made and I kill 100, will it notice that 100 have been "pre-decimated" and work it's way up to 900 strats before decimation occurs, or will it keep working until there's a total of 1000 strats in the databank before decimation happens?

it is not possible to manually step-in into the decimation process, but you can do it a different way. You can use the Random generation first to make enough strategies - in this step you can let it run for as long as you'll find enough good strategies. Then you'll start Genetic Evolution over these new strategies. This way you'll have total control over the initial population and you can bypass decimation.

I've been tweaking my Evolution settings. I've noticed that if I run 100+ generations, evolution has a bad habit of getting stuck in a rut and I still haven't quite been able to keep it from doing that. By this, I mean that certain strategies with tiny variations take over and there will be a dozen or more strategies with only tiny difference, then another set of near identical results. Maybe I should increase the mutation factor? That would result in more bad results, but should also encourage more variety.

yes, it happens that Evolution goes into the dead end. Increasing mutation factor might help, but you'll influence evolution much more by increasing Normalization Coef (in Strategy Ranking Options). The higher Normalization Coef, the smaller influence of strategy ranking, which effectively means that the best strategies are chosen with slightly less probability, resulting in more diverse population, without the very few best strategies taking over entire pool.


There is a list of changes in every update, you can see it here:
http://www.GeneticBuider.com/whatsnew

 

[Pharaoh]

Hi Mark,

Good to hear there will be some additional features. Thanks for some of the other suggestions.

Hmmmm... Got 1200 strats generated randomly sitting in the database now. I kept going in every few hours and dumping the lame ones that somehow had high fitness scores. I'm going to take the best of these and my other "A" strategies and see what genetic evolution does to them.

I'll play with the normalization coefficient. The other thing I need to spend more time on is tweaking the fitness rating. I keep getting these strategies that claim high fitness, but I sometimes see more stability and profit along with less drawdown strats with slightly lower fitness ratings.

As long as I'm writing, what's up with the drawdown percentages? I see some that claim infinite drawdown, but the drawdown doesn't look too severe to me. Does this thing assume a certain account size and lot size?

I can see moving some strats over into MT4 in the not too distant future for backtesting and demo testing. Does the code generated handle both 4 and 5 digit brokers? Also, since time and day of the week are items usable in strategy generation, will the EAs include a manual GMT offset or automatic detection of this?

One other suggestion. Since move SL to BE is already included, how about adding a "close 1/2 of the position and move SL to BE" option? Alternatively, just a "closed 1/2 of position" option that could be combined with moving SL to BE and/or closing that half at TP and trailing the stop after the other half.

 

[Pharaoh]

I'm trying a new method of getting rid of those ones with losses and no trades out of sample. I set up a random generation and told it to select by OOS profit instead of fitness. I'll still need to manually dump a lot of things, but this should at least eliminate those nice, stable ones that stop trading or have small losses on OOS data.

 

[Mark Fric]

just a note - random generation is a good mode on its own, it is not just a step before using evolution.

I'll play with the normalization coefficient. The other thing I need to spend more time on is tweaking the fitness rating.
I keep getting these strategies that claim high fitness, but I sometimes see more stability and profit along with less drawdown strats with slightly lower fitness ratings.

I personally put most weight on Stability rank. Stability tells you how smooth is the equity curve, which means also lower drawdowns.

As long as I'm writing, what's up with the drawdown percentages? I see some that claim infinite drawdown,
but the drawdown doesn't look too severe to me. Does this thing assume a certain account size and lot size ?

drawdown computation is quite difficult and there are many edge cases, I'll look at the algorithm.
Maybe the problem is in strategies that start with loss in the beginning.

can see moving some strats over into MT4 in the not too distant future for backtesting and demo testing. Does the code generated handle both 4 and 5 digit brokers?
Also, since time and day of the week are items usable in strategy generation, will the EAs include a manual GMT offset or automatic detection of this?

yes, the code handles all brokers, 4,5 digits, also ECN/non-ECN. There's no GMT offset detection yet (idea for a feature), strategy simply expects that the GMT offset
of your MetaTrader will be the same as of the test history data. But you can easily change the hours in EA parameters according to your broker.

One other suggestion. Since move SL to BE is already included, how about adding a "close 1/2 of the position and move SL to BE" option?
Alternatively, just a "closed 1/2 of position" option that could be combined with moving SL to BE and/or closing that half at TP and trailing the stop after the other half.

I was thinking about this, but it will add a lot of complexity to the strategy. There are very many options of partial profits and Genetic Builder is a tool to generate trading rules
not rules of open positions management.

I'm trying a new method of getting rid of those ones with losses and no trades out of sample.
I set up a random generation and told it to select by OOS profit instead of fitness. I'll still need to manually dump a lot
of things, but this should at least eliminate those nice, stable ones that stop trading or have small losses on OOS data.

I think you misunderstand the function of OOS, it has no real sense in Random generation mode, because there's no evolution.
In Genetic Evolution mode only in-sample data are used to compute fitness, which is then used in evolution. The out-of-sample is used to test the performance on data that are not visible by evolution.
But with random generation, OOS will just split the chart to two parts, but otherwise it does nothing.

When I generate strategies using random generation, I do it in two steps:
1. start generation on 60-70% of the history data (I don't use OOS at all)
2. after enough candidates are found retest these strategies on the rest of data and dismiss all that don't perform well

If the strategy is robust, it must be profitable also on the unknown part of data.
This two step process helps you filter out curve-fitted strategies, which is the biggest danger with automatic generation.

 

[Pharaoh]

Thanks for answering all the silly questions. I'm trying to put your program through maximum possible abuse.

Now all I need is 4 or 5 computers to run it in parallel so I can see how the options vary.

I've got a multi-day random generation going (1200 in databank) and told it to sort by OOS profit (yes, I know you do the OOS separately, but this looks like it saves a step). A large percentage of all the strategies had zero trades in OOS, so those are slowly being chewed away (the negative ones are already gone). Every now and then I go in and check the profitable ones and toss some of the profitable ones that have undesirable characteristics.

How is stability calculated? I've seen some with reasonable stability rankings that didn't have large drawdowns, but that parked themselves in small drawdowns for extended numbers of trades.

 

[Pharaoh]

If a million code monkeys typed random trading algorithms for a million years, one of them would create the ultimate Holy Grail EA.

One thing I love about Genetic Builder is that it provides me with the coding power of a few hundred thousand code monkeys that can all type very fast. I'd need more power to get the Holy Grail, but some of the EAs it's created are pretty impressive, and I still haven't checked out all the possible options.

I'd like to explore the question of doing OOS analysis only after random generation vs. doing it as part of random generation a bit further.

Assume I generate a random strategy using part of the data for In sample and blocking out the rest of the data. I like the strategy, so then retest it using the same In sample and open the rest of the data up to be OOS.

Alternatively, I generate the same strategy (ok, the same strat won't be generated, but for sake of simplicity, assume that it is) with both In and Out of Sample divided the same way, but let OOS be displayed as part of the initial generation.

Shouldn't the final graph showing both in and out of sample equity be identical in this hypothetical case?

I'm hoping that it is. Currently, I'm running a second multi-day random generation and telling it to select results by OOS profits. every now and then I sort by OOS profits, delete anything with too low of stability, then look at the ones with good stability scores. The impressive ones get saved.


I still haven't played with the options to try to optimize strategies. That's coming up soon on my to-do list.


The really good news is that if I can find a broker either with really narrow spread or one that gives good rebates, I've found a large number of strats that have fairly flat equity curves. Some go into drawdown for a couple hundred trades, but eventually make a modest profit. Narrower spreads or spread rebates could turn these into something very profitable.

One other question. Since this shows profits and losses in dollars, a certain size for trades must be assumed. Is it operating like it's trading full lots ($10/pip of xxxUSD) or some other amount?

 

[Mark Fric]

How is stability calculated? I've seen some with reasonable stability rankings that didn't have large drawdowns, but that parked themselves in small drawdowns for extended numbers of trades.

Stability is calculated from the equity curve, generally the smoother the equity curve (with smaller extremes like Drawdowns), the bigger Stability number.
It is computed using deviations from "ideal" curve, plus some weight is added to the curve slope.

I'd like to explore the question of doing OOS analysis only after random generation vs. doing it as part of random generation a bit further.

Assume I generate a random strategy using part of the data for In sample and blocking out the rest of the data. I like the strategy, so then retest it using the same
In sample and open the rest of the data up to be OOS.

Alternatively, I generate the same strategy (ok, the same strat won't be generated, but for sake of simplicity, assume that it is) with both In and Out of Sample divided the same way,
but let OOS be displayed as part of the initial generation.

Shouldn't the final graph showing both in and out of sample equity be identical in this hypothetical case?

yes, you are right, the results and charts will be identical

I'm hoping that it is. Currently, I'm running a second multi-day random generation and telling it to select results by OOS profits.
every now and then I sort by OOS profits, delete anything with too low of stability, then look at the ones with good stability scores. The impressive ones get saved.

Problem of this aproach is that it doesn't ensure that the strategy isn't curve fitted to the data.
You generate strategies using all available data, which are divided into IS and OOS part. Then you'll use performance on OOS part to filter the good strategies.
But it is still possible that the results are good only because the strategy was curve fitted to the (IS or OOS) data.
Curve fitting can be revealed only by testing the strategies on unknown data.


Two steps approach as I described before ensures that curve fitted strategies are filtered out, because:
1. you first generate strategies on part of the data, and choose only the ones that were profitable
2. you then retest the strategies on unknown part of data and choose the remaining ones that are still profitable.
If the strategy was curve fitted to the data from step 1. it would fail in the second step.

Strategies that pass these two steps are for sure not curve fitted to the data.
I hope you see my point.

One other question. Since this shows profits and losses in dollars, a certain size for trades must be assumed. Is it operating like it's trading full lots ($10/pip of xxxUSD) or some other amount?

yes, it trades with 0.1 lots, so the results are actually in pips for all the pairs

 

[Pharaoh]

Minilots. . . . dollars equal pips . . . good idea. That does simplify a lot of things.

I see your point about curve fitting. I think that's a risk whether I include OOS upfront or not. My plan is to retest my best on another time frame and/or another pair. The limited testing I've done like this turned one of my B-graded ones into an amazing A-graded one. It also wiped out some of my A-graded ones.

I need to get more data. Trying to download minute date from my broker returned a disappointingly small amount of data. I'd love to get the complete range from the end of what comes with Genetic Builder to current. I'd use that for an "extended OOS" to do followup testing on the ones that thrive under all the initial conditions.

 

[Pharaoh]

Woohoo! Got an early Birthday present from Mark. I now don't have to worry about that license expiring. THANKS MARK!!!!


I found a workaround to the issue of Windows automatic updates triggering reboots. I disconnect the system from the internet just before starting a multiday build. When I've filtered out everything and saved the best strats, I reconnect, update my antivirus software, and then run a windows update before rebooting.

One feature request (not a priority, but would be nice):

A filter feature like: Delete all strategies with _____ above/below ____

First blank would be a dropdown with choices like fitness, stability, max drawdown, etc. Second one would be a number. Above/Below could be dropdown or radio buttons. System should display the query, number of strats about to be deleted, and ask user to proceed or cancel (like the current delete of checked strats).

This would make dumping useless strategies (low stability, high drawdown, etc.) much faster.

 

[Mark Fric]

Hi Pharaoh,

I have a gift for everybody. I'm publishing EURUSD strategy from my Portfolio 1 with complete source code for free here.
The strategy is based on ADX, and it is very robust because it works also on other timeframes and even on other currencies (other ADX period or SL/PT values should be probably used).


As for the feature of filtering results, it will be solved in a better way in the next version.