How to Evaluate an Automated Trading System
Anyone can describe an automated trading system in glowing terms. What matters is whether the claims hold up and whether the system suits you. This page is a checklist for looking at any automated strategy, ours or anyone else's, before you put real money behind it. It is part of our guide to NinjaTrader® automated strategies.
1. Are the rules clear?
You should be able to say, in plain words, what the system is trying to do: what market it trades, in which hours, and what kind of price behavior it looks for. If the seller cannot or will not explain the idea, treat that as a warning sign. Trading systems differ widely in style, and the explanation tells you whether the system matches how you want to trade.
2. How was the performance measured?
Performance numbers come in very different kinds, and they are not interchangeable:
- Hypothetical (back-tested) results come from running the rules on past data. They are produced with hindsight and no real orders, so they have limits that real trading does not.
- Simulated results come from running the system live in a practice account.
- Actual results come from a real account, with real fills, real commissions and real slippage.
Ask which kind you are looking at. Reputable vendors say so plainly, and regulators require disclosures about hypothetical results. Our page on backtesting and curve fitting explains why hypothetical numbers need care.
3. Is there enough data behind the numbers?
A handful of trades proves very little. Look for results that cover many trades and different market conditions: trending, choppy, quiet and volatile. Ask how the number of trades compares with the number of settings the system has. Systems with many adjustable settings are easier to fit to the past.
4. How big were the losses, not only the gains?
Do not stop at the profit figure. Look at the largest drawdown (the drop from a peak to a later low), the length of the losing streaks, and the size of the largest losing trade. Then ask yourself honestly whether you could keep a system running through a drawdown of that size, and whether your account could absorb it.
5. Are costs included?
Commissions, exchange fees and slippage (the difference between the price you expect and the price you get) reduce results, and they matter most for systems that trade often. Check that any numbers you are shown include realistic costs.
6. What does it take to run it?
- Platform and data: which platform, which instruments, and which data feed?
- Account size and contract size: what is the smallest account the vendor considers sensible, and how many contracts does the system trade?
- Your own time: does it need to be watched, restarted each day or run on a dedicated computer?
7. Test it yourself before you trade it
Run any strategy on a simulated account first, for long enough to see how it behaves on your computer, with your data and your broker. Our guide on running an automated strategy on NinjaTrader® 8 shows how.
Looking at our strategies
Each of our strategy pages describes what the system does and what it trades. Compare them against this checklist: NinjaTrader® strategies and TradeStation® strategies.
Futures trading contains substantial risk and is not for every investor. An investor could potentially lose all or more than the initial investment. Hypothetical performance results have many inherent limitations. No representation is being made that any account will or is likely to achieve profits or losses similar to those shown. Past performance is not necessarily indicative of future results.