What people usually mean by testing
"I tested it for two weeks and it did not work."
Two weeks on one instrument, in one session, during whatever the market happened to be doing. That is a sample of one market condition, measured once. Whatever came out of it describes those two weeks. It may or may not describe the tool.
This is not a reason to skip testing. It is a reason to know what the test is capable of telling you, which is less than it feels like at the time.
The sample is not the number of trades
Thirty trades sounds like a sample. It is not, if all thirty happened in the same conditions.
The things that change the answer, and that most tests hold constant without meaning to:
Regime. A trend-following tool tested during three weeks of trending market will look excellent. The same tool, the same code, the same settings, tested in a range looks broken. Neither result is about the tool.
Session. The same instrument behaves differently at different hours. A method that works in one session can be a slow bleed in another, and testing in only one tells you nothing about the other.
Day of week. Mondays and Fridays are not Wednesdays. Position squaring, weekly opens, the shape of liquidity — these are real and they are small enough to disappear in a thirty-trade sample while still moving the result.
News. A scheduled release inside the test window can produce two or three outliers that decide the total. Including them is correct — they happen. But a result built on three outlier days is a result about three days.
If you take one thing from this note: a test tells you how something behaved in the conditions you tested. Extending that to "it works" or "it does not work" is a claim about conditions you have not seen.
Ten losses in a row
This is where most tools get abandoned, and there are three distinct explanations that look identical from the inside.
| What happened | How to tell |
|---|---|
| The tool does not have an edge | The losses are spread across different conditions. It fails in trend, in range, at the open, at midday |
| The tool is fine, the regime is wrong | The losses cluster. Nine of the ten came from the same kind of market, and the earlier winners came from another kind |
| The tool is fine, the execution was not | The entries matched the rules, the exits did not. Or the size changed. Or some were taken early |
Only the first is a reason to drop the tool. The second is a reason to add a condition about when to switch it off. The third is not about the tool at all.
Telling them apart requires one thing: a record made at the time. Not a memory. Ten losses recorded with the market condition next to each one answers the question in a minute. Ten losses remembered answer nothing, because by then you remember them as a run of bad luck or a broken tool, depending on your mood.
"The signals were good but I still lost"
This is a common and genuinely confusing outcome, and it usually has a dull explanation.
An entry is one of four decisions. Where you get in, where you are wrong, where you get out, and how much you take. The signal addresses the first. The other three decided your result.
In the first note on this site I measured a tool of mine at a twenty-nine percent hit rate, consistent across three timeframes. At a 1.5R target that is a losing method. At 2.5R it barely signalled. The entries were identical in both cases. If I had looked only at the entries, I would have concluded the tool was fine — and kept losing.
So when the signals look right and the account does not agree, check the other three decisions before touching the tool. Most of the time the answer is there, and it is not interesting, which is why people prefer to look for a better indicator.
The question almost nobody asks
Did you follow your strategy while using the tool — or did using the tool break it?
This is subtler than discipline. Suppose your method is patience: few trades, long waits, one or two decisions a week. You add a tool that produces six signals a day. You take three of them because they looked good. You have not broken a rule exactly — each one matched something. But you are no longer running the method you tested. You are running a different method with the same name.
The tool did not fail. It changed you, quietly, by making activity available.
This is the single most common way I have seen a working approach stop working. Not a bad signal. A good signal, offered more often than the method could absorb.
Before judging a tool, it is worth asking whether your trade count went up while you were testing it. If it did, you measured a different method.
How long is long enough
There is no number that is correct in general, but there are useful floors.
Enough instances to count, not enough to tune. Fifty is far better than fifteen. But if you adjust settings during the fifty, you no longer have fifty — you have a description of the settings you landed on.
At least two different regimes. If the whole period trended, wait for a range before concluding anything. This usually means a test takes longer than you want it to, and that is the honest cost.
One change at a time. Change one setting, then observe. Changing two means you will never know which one did it, and you will keep both out of superstition.
Write the expectation down first. Before the period starts, say what you expect to see. "I expect it to work in trend and get chopped up in a range." That sentence, written in advance, is what stops you from reading the result you wanted out of an ambiguous outcome.
There is no good or bad indicator
An indicator does what it was programmed to do. If the code says "mark when the close is above an average of the last fifty closes", that is what it marks, every time, without opinion. It does not know what instrument it is on, what hour it is, or what you are hoping for.
Everything else — whether that mark is useful, whether this is the right moment, whether the position should be half the size — happens outside the script. That is not a weakness in the tool. It is the division of labour.
Which means the question "is this indicator good" has no answer. The answerable questions are narrower:
Does it do what its description says? There are nine questions on this site for that, and one of them takes two minutes.
Does it answer a question I actually have, or one I already have covered?
Does it fit the way I can trade — the hours I have, the patience I have, the losses I can take without the next decision getting worse?
Do I know when to switch it off?
That last one is the mark of someone who has used a tool long enough to understand it. Knowing when a tool is wrong is more valuable than knowing when it is right, because the first one is rarer and nobody sells it.