How to Optimize an Expert Advisor (EA) on Metatrader (MT4 | MT5) for Live Trading.
Due to excessive marketing and hype created by marketers, Expert advisors (trading robots) have gained a bad reputation in the industry.
In reality, an expert advisor is just a piece of software that contains the instructions to make the trades decision based on a trading strategy.
If you have a trading strategy that you think might succeed in the market, you should definitely get it converted (or code it yourself) into an expert advisor for the following reasons:
- Ability to backtest: For manual traders, it is tough to find what parts of their trading strategy work and what doesn't work. With Algo-trading using an expert advisor, they can run their trading on past data to see if it would have worked in the past.
This is one of the significant advantages of automated trading over manual trading as it helps to find and remove any flaws of a trading system before going live with the system.
- Accuracy: Compared to an expert advisor, in manual trading, it is much more possible to buy or sell the wrong currency pair or for the wrong lot size. Pre-written algorithm can make sure it double-check the order details before sending it to the broker's server.
- Speed: One of the main reason is speed. Since EAs are written beforehand and executed automatically, the rate at which trades are made is nearly impossible for a human to perceive.
- Reduced transactions cost: Compared to a manual trader, an EA can place the trades at a much faster rate that often leads to lesser transaction costs.
- Eliminate Human Emotions: Trading robots eliminate human emotions and behavioral problems like exiting too early or too late from a trade.
Despite having so many advantages over manual trading, traders are still skeptical about the expert advisors. Hardcore marketers and scammers have been promoting them like a magic formula in a black box, which will make you rich over months if not overnight.
So if we remove the marketers and scammers from the situation., still why many of the expert advisors fail in live trading?
The answer is..... Either under-optimization or over-optimization.
Financial markets are dynamic and continuously changing. An EA should be optimized at least once a year to adjust the trading strategy according to the new market conditions.
In this article, we briefly explained how to optimize an expert advisor and prepare it for live trading.
What is EA optimization?
Optimizing an expert advisor means to find the input parameters in EA that maximize the profit or any given criteria (such as Sharpe ratio).
For example, if you have a moving average crossover system, it buys when fast moving average cross above the slow moving average, and sells when fast moving average cross below the slow moving average.
Using the EA optimization, you can find the optimal values for the fast and slow-moving averages that leads to maximum profit within a fixed historical period.
What is Curve Fitting?
Curve fitting in trading is the process of designing a trading system that adapts so closely to historical data that it loses its predictive power and becomes ineffective in the future.
EA optimization is both science and art. If you over-optimize, the system will be susceptible to curve fitting and lose its predictive power in live trading, and if you under-optimize, the EA will under-perform.
In a nutshell, Optimization means “to make fit” a system; that is, to adapt a system to the market we intend to trade.
To demonstrate the complete EA optimization process, I will optimize the KT CCI Divergence EA on EURUSD 1-Hour.
Curve fitting is a by-product of EA optimization. We cannot avoid it entirely, but we can surely minimize it by following a particular optimization sequence and optimizing each input parameter separately.
We can divide the input parameters of CCI Divergence EA into three main sections:
Backtesting result with the default inputs:
Even with default input parameters, the strategy has shown some signs of profitability. But it is not desirable to trade the strategy having such a massive drawdown of 88%.
Before we dive into the optimization process, we must understand about the objective function.
According to Wikipedia, an optimization problem consists of maximizing or minimizing a real function by systematically choosing input values from within an allowed set and computing the value of the function.
So looking at the strategy results above, what value you would choose to improve?
If you choose to optimize the net profit value, there is a possibility that the value of net profit would improve, but drawdown value will not come down.
If you choose to optimize the profit percentage, it is very much possible that the profit percentage would increase up to 90%, but net profit will come down to a negative value (profit percentage is highly overrated in quantitative trading).
So what objective functions you would choose to improve so that it supports and enhance the overall strategy performance?
To solve this problem, we have incorporated a custom objective function in all of our expert advisors. When you choose to improve this custom objective function, if not all, but most of the time, overall strategy results will improve.
Setup the strategy-tester in Metatrader for optimization:
Optimizing the CCI period:
Creating a 2-D graph using Excel
To find a stable value among the optimization data, a 2-D graph must be created.
CCI Period 2-D graph:
Changing the CCI period to 46 provides the following result:
The equity line has improved a lot. A simple change of CCI period from 14 to 46 has increased the net profit by 280% while reduces drawdown to 27% from a massive drawdown of 88%.
This is an astonishing improvement considering the fact that currently, we have not added any filters or stop-loss/take-profit.
You may be wondering why I've not chosen a CCI period of 50, where we would have got the maximum value of our optimization function along with the maximum net profit?
Instead of choosing a maximum outlier, it is always better to select a stable value that has a good number of adjacent members.
The neighborhood of your chosen parameter must be nearly as profitable as your selected system parameter, and the bigger this profitable parameter range is, the better.
"If you don't like the neighboring numbers, you've got a problem, because odds are, you will wind up with the results of the neighboring set of parameters."
- Murray Ruggiero, professional trading system developer.
We have successfully optimized the entry parameter. Now we can proceed ahead with adding filters...
Instead of optimizing a single value, now we are looking to add appropriate filters using the optimization feature in Metatrader.
Finding a stable value within the parameter space is not applicable in this case. So, I've chosen an optimized value that provides reasonable improvements with a good number of trades so we can have some statistical advantage in the trading system.
After adding the suitable filters, we've got the following result:
Adding the suitable filters has slightly increased the net profit but significantly decreased the drawdown from 27% to 13% while boosting the profit factor from 1.21 to 1.50.
Moving forward to the final and last stage of our EA optimization, in which we will find and add the suitable stop-loss and take-profit.
Instead of using the fixed number of pips for SL and TP, we'll use the ATR (Average True Range) to make the calculation proportional to the market volatility. At Keenbase-Trading, each of our EA provides a choice to set the stop-loss and take-profit using the market volatility.
Stoploss 2-D graph:
Take-Profit 2-D graph:
After adding the SL and TP, we've got the following result:
Move the slider left and right to compare the before and after optimization equity graph:
Comparing the final result with the initial result:
- The net profit has increased by 334%.
- One of the significant improvements we got after the ea optimization is that the drawdown has reduced to just 9% from a high value of 88%.
- The number of total trades has reduced to 685, which is still a good number having a statistical significance.
- The profit percentage has increased to 67% from 54%.
- The profit factor has increased to 1.58 from 1.06
- The whole optimization has been done using a fixed lot size of 0.1 per trade, using money management such as a fixed fractional method will further boost the net profit and provide exponential equity growth.
- Advanced traders can validate the final model using some stress testing like Monte Carlo simulation, out of sample, or walk-forward analysis.
- After optimization, it is advisable to closely monitor the performance of the model in a live market using a demo account or a small real account for at least six months.