Walk-Forward Tests

What a Walk-Forward Test Is

A walk-forward test simulates periodic re-optimization. The overall date range is divided into a series of intervals, and for each interval the strategy is run using the best parameter values found in the previous interval. Rather than measuring how a single fixed set of parameters would have performed, it measures the effectiveness of the process of re-optimizing at regular intervals.

Two Ways to Create One

A walk-forward model can be generated either from the Optimizer dialog or entirely in the script with an OptimizeSettings section. Either way, an Optimize run appends a WalkForward section to the script, which RealTest then replays whenever the script is run in Test mode. The settings that turn an ordinary optimization into a walk-forward are the same in both places:

In the dialog	In OptimizeSettings	This example
Optimization Mode	OptimizeMode	Combinatorial
Score	OptScoreCol	NetProfit
Time Unit	OptTimeUnit	Days
Test Length	OptTestLength	252
Test Interval	OptTestInterval	63
Keep Best	OptKeepBest	1
By Date	OptKeepByDate	True
Create Walk-Forward	OptWalkForward	True

The other dialog options simply control the optimization itself; the items above are what make it a walk-forward.

Setting It Up in the Optimizer Dialog

This walkthrough uses the sample2.rts script from Tutorial 2.

Open that script again and click on .

Now make all of the following selections in the Optimizer dialog:

1.Parameters — Check both parameters.

2.Optimization Mode — Set to "Combinatorial"

3.Test Iterations — Set to 1.

4.Score — Set to NetProfit, or any other Results Window item (statistical formula) you want to optimize for. This is your "Fitness Function".

5.Clear Results Window — Check this so any prior tests will be deleted, and/or open a new results window.

6.Sort After Each Test — Check this whenever generating a Walk-Forward test.

7.Keep Best — Set to 1 for Walk-Forward.

8.By Date — Check this (it will become enabled after the next step if it isn't already).

9.Time Unit — Set to Days (or a longer unit if desired).

10.Test Length — Specifies how many days (since that's the Time Unit) to include in each backtest (252 days is one year).

11.Test Interval — Specifies how often to re-optimize (63 days is one quarter).

12.Create Walk-Forward — Check this.

13.Maximum Test Count — This will show how many tests need to be run. (The current version of the example script has different parameters so the count will be different.)

Now you are ready to press and let RealTest generate your walk-forward model.

Though there are about 5,000 tests to run, it will probably take less than one minute to finish, depending on the speed of your computer.

As the tests run, you will see the Results window being updated continually. To help the process finish faster, minimize the Results window until all tests are finished, then restore it again.

Here is what is going on under the hood:

for each 63-bar interval of the overall date range

for each of the 40 possible combinations of the two parameters

run a 252-bar test and keep the result if it is better than the prior best result

record the start date and parameter values that produced the best result

Once the above is complete, a new WalkForward section is automatically added to your script. This section contains an item called "Dates", and an item for each of your parameters. These things together serve to define how to run the final Walk-Forward test, in which the best parameters from the previous interval are used for each 63-bar interval. In effect, this becomes a test of the effectiveness of the process of periodic re-optimization.

Here is a snippet of how the results and script windows will look at the end of the run:

The Results window shows the best "in-sample" test for each interval. The final two tests run are the in-sample (1480) and out-of-sample (1481) results for the entire date range.

The script window shows the new WalkForward section that was generated. These three rows extend far to the right. The first row contains the start date of each interval and the subsequent rows contain the best value found for each parameter in each interval.

To run the walk-forward test again (without re-generating all the parameter values), simply press the Test button or select Run Test from the Run Menu:

When a script includes a WalkForward section (and it's not commented out), RealTest will always use it when running that script in Test mode.

Setting It Up in the Script

Rather than filling in the Optimizer dialog, you can define the entire walk-forward in the script by adding an OptimizeSettings section. The block below reproduces the dialog selections shown above for sample2.rts, with one addition:

OptimizeSettings:
   OptimizeMode: Combinatorial
   OptScoreCol: NetProfit
   OptTimeUnit: Days
   OptTestLength: 252
   OptTestInterval: 63
   OptKeepBest: 1
   OptKeepByDate: True
   OptWalkForward: True
   OptNoDialog: True

With that section present, run the script in Optimize mode (Run / Run Optimization). Because the block sets OptNoDialog: True, the Optimizer dialog is skipped and the run starts immediately; the same WalkForward section is generated and appended, after which running in Test mode replays it exactly as before.

Leave OptNoDialog out (or set it to False) if you would rather have the Optimizer dialog open pre-filled with these settings, so you can review or adjust them and confirm before launching the run.

The factory example spy_tlt_uis.rts is a complete walk-forward strategy set up this way. Open it from the Examples folder to see an OptimizeSettings section and the WalkForward section it generates in a working script.