CSI Futures Data Import

CSI Futures Import

For end-of-day futures data, CSI has been the most respected source for many decades.

RealTest fully supports CSI's futures data and makes it easy to import and work with it.

To use CSI data with RealTest, you must have downloaded and installed their Unfair Advantage software and subscribed to one of their futures data packages.

Within the Unfair Advantage program, create one or more new futures portfolios for use with RealTest.

1. Click on "New Portfolio":

2. Provide a name and folder location for the portfolio's CSV data files and set some properties of the CSV files to create:

It does not matter what the folder name and path are, just that you remember them so your RealTest script can point to it.

There are a large number of portfolio settings spread across 11 window tabs here.

The first two rows of tabs can be kept on their defaults.

The fields under "ASCII/Excel Files" (by which they mean CSV files) that are important for RealTest are:

a. the file names for individual contracts should use the S_CYA format, e.g. VX_2024Z

b. the "Make TradingBlox Compatible" box must be checked so that UA will create a "futuresinfo.txt" file

3. Specify the field-level details of the CSV export files:

The key here is that the field list includes at least the following items:

•D - date

•O - open

•H - high

•L - low

•C - close

•U - unadjusted close

•X - expiry date

•N - delivery number (delivery year and month)

It does not matter what order these appear in, just that they're all present.

If included, RealTest stores the unadjusted close price in the Extra field of each bar.

If included, RealTest stores the expiry date and delivery number as an Event records for the dates on which they changed (roll dates).

Expiry date is accessible via InfoExpiry or Event(-999), delivery number is accessible via Event(-998).

Expiry date is used to know when to add roll costs to a backtest and to specify the correct contract in CSV order lists.

Delivery number (year and month) is used to show the correct human-readable symbol (e.g. NG_2025F) in test logs and text order lists.

The above set of fields leaves one slot available in the RealTest bar records: Volume. You can use any of V, v, I, or i (total volume, contract volume, total OI, contract OI) as your volume field as desired, or leave it out if you don't need it. Whichever one of these is encountered first (left to right) will be used as Volume.

4. Specify the remaining details of your CSV files:

The "no record for holidays/weekends" settings are recommended but not required.

The "Include a Header Record" setting is required if you do not want to have to provide a CSVFields statement in your Import definitions.

5. Add symbols to your portfolio:

a. Click on "Add Symbol" in the main window

b. When this dialog appears, select the "Back-Adjust" tab

c. Configure your rolling criteria, timing, etc. (above is just an example -- do it however you prefer)

d. Type a symbol root or click on Market Specs to select one or more

e. Click OK

6. Once your portfolio is configured, UA will already have exported your CSV data files to your specified folder, and you can now import them into RealTest.

7. To keep your date up to date every day, simply run the "CSI EZ Download" program. It automatically downloads all the latest data to their internal database and then updates all of your portfolio files.

8. To import your CSI data into RealTest, the Import definition will look like this:

You can add more than one DataPath if you want to combine data from multiple CSI portfolios into a single RealTest RTD file.

You can optionally name each DataPath using a comment like {"CME"} and later refer to it using e.g. InList("CME") if your formulas need to know which portfolio the current symbol came from.

As with CSV Import, you can optionally use IncludeList and/or ExcludeList. If you use IncludeList, this means "only include these symbols and exclude all others" (unless you also specify KeepNonIncluded: True). ExcludeList means "include all others except these".

Also as with CSV import, the file name becomes the symbol. Most often with CSI Futures, file names look like xx_b, where xx is the root symbol of the futures market and _b means "backadjusted". You will refer to those symbols as needed using e.g. Extern($cl_b, close).

Note that when you generate orders using this data, the symbols used in the orders represent the specific contract to be traded even when using the back-adjusted CSI data without individual contracts included.

Note also that with CSI Futures, you do not need to provide a SymInfoFile. RealTest reads and processes the futuresinfo.txt file found in each DataPath folder to get the market name, point value, and tick size for every symbol. This is why it's required to check that box in step (2) above.

See Using CSI Futures Data for details about using this data in your backtests.