JTOOLS TIME SALES FILE GENERATOR - Gen II


JTools Table of Contents


Project Location: \JTools_TSFileGeneratorG2

Born from the original TimeSales File Generator application, the 2'nd generation file generator has combined Batch Processing and richer file outputting options for a powerhouse of raw Time and Sales generation.

Like the original TimeSales File Generator, Gen II allows you to dump time and sales reports for any single symbol (or multiple symbols) in almost any fashion you desire. Every field available for NxCore Trade, ExgQuote and MMQuote messages is offered as possible output. Fields generated and field order are determined by the usage strings defined by the user.

The JTools Time Sales File Generator also combines the functionality of the JTools Batch Processor to make processing data which spans days, months or years trivial.




Symbol Parameters:
  • Symbol - Symbol(s) for which data is to be output. The symbol(s) may be entered in one of the following manners:

    - A single symbol, such as " eAAPL "
    - Multiple, comma delineated symbols, such as " eAAPL,eIBM,eGOOG,eSPY "
    - A wildcard symbol, such as " eA* "

    See also "Considerations: Multiple Symbols/File Limitations" below.

  • Read Symbols From File - If checked, symbols of interest will be read from a text file of choice.

  • Sym File - If reading symbols from a file, this will be the file the application uses.

    See also "Considerations: Multiple Symbols/File Limitations" below.

  • Listed Exg - Limit symbols to only those on the specified listed exchange. You may enter a single exchange, multiple comma delineated exchanges, or leave the field blank for all exchanges.

  • Reporting Exg - Limit output to data from a specified reporting exchange. You may enter a single exchange, multiple comma delineated exchanges, or leave the field blank for all exchanges.

Time Period Parameters:
  • Start Time - Start Time (HHMMSS). Example - "093000".

  • Stop Time - Stop Time (HHMMSS). Example - "160000".

Data Parameters:
  • Trade Data Include Fields - A sequential string with the fields desired for output of trades. The order of output in output files is determined by the field order specified.

  • Show Trades - If checked trades are recorded and output to files.

  • Exg Time Format - The time resolution to print exchange timestamps at. HH=Hours, MM=Minutes, SS=Seconds, MMM=Milliseconds, mmm=Microseconds. ( Note that Milliseconds are available only with tapesets nx2 and nx3 and Microseconds are available only with tapeset nx3).

  • Exg Quote Data Include Fields - A sequential string with the fields desired for output of exchange quotes (level 1 quotes). The order of output in output files is determined by the field order specified.

  • Show Quotes - If checked exchange quotes are recorded and output to files.

  • Split Bid/Ask - If checked then bid and ask quotes will be printed on seperate lines. If unchecked both bid and ask quotes are printed in a single line.

  • Exg Time Format - The time resolution to print exchange timestamps at. HH=Hours, MM=Minutes, SS=Seconds, MMM=Milliseconds, mmm=Microseconds. ( Note that Milliseconds are available only with tapesets nx2 and nx3 and Microseconds are available only with tapeset nx3.

  • Depth (MMQuote) Data Include Fields - A sequential string with the fields desired for output of depth quotes (level 2 MMQuotes). The order of output in output files is determined by the field order specified.

  • Show Depth - If checked depth quotes (MMQuotes) are recorded and output to files.

  • Split Bid/Ask - If checked then bid and ask quotes will be printed on seperate lines. If unchecked both bid and ask quotes are printed in a single line.

  • Exg Time Format - The time resolution to print exchange timestamps at. HH=Hours, MM=Minutes, SS=Seconds, MMM=Milliseconds, mmm=Microseconds. ( Note that Milliseconds are available only with tapesets nx2 and nx3 and Microseconds are available only with tapeset nx3. Additionally, for MMQuotes only futures contain milliseconds and microseconds).

Misc Parameters:
  • Print Header - If enabled a header will be written to the first line of each files indicating the field names and field order.

  • Exg Codes as Text - If checked, exchange codes are output as text, otherwise they are output as numeric.

  • Cond Codes as Text - If checked, condition codes for trades and exg quotes are output as text, otherwise they are output as numeric.

  • Milliseconds in NxCore Timestamp - If checked milliseconds for all NxCore timestamps will be output, otherwise only hour,minute and second are output.

  • Decimal precision - ("0" - "5", "-1" for auto based on price type).

  • Process Saturdays - Process data for Saturdays toggle.

  • Process Sundays - Process data for Sundays toggle.

  • Output Symbol Prefix - If checked symbol prefixes will be output with the symbol.

  • Equity Option Subfields - If checked, equity and index option symbol will contain subfields for PutOrCall designation, expiration date and strike price. The fields will immediately follow the symbol. Symbols must be specified in the include fields for the sub-fields to be generated. The field order will be as follows:

    Option Symbol|PutCallDesignator|ExperiationDate|StrikePrice|

    and The fields will be separated with the sub-field separator character.

Output File Parameters:
  • Separate Output Files for Trades, Quotes, Depth Quotes - If selected, separate output files for trades, quotes and depth quotes will be generated (and labelled as such). If not selected, all date will be output to the same file.
  • Use Symbol in Filename - If selected, output files will be separated (and named) by specific symbols.

  • Append Multiple Days to a Single File - If selected, data for multiple days will be written to a single file, as opposed to a dated file per trading day.

    See also "Considerations: Appending Multiple Days to a Single File" below.

  • Default File Extension - The default file extension to use for filenames.

  • Field Separator - A single character used to separate data fields within the files.

  • SubField Separator - A single character used to separate sub-fields within primary fields. Currently sub-fields will be present in Extended Trade Conditions and Equity/Index option symbols (when Equity Option SubFields is specified).

  • Output Data Folder - Folder where data files are to be written to. If the specified folder does not exist the application will attempt to create it.

  • Delete Existing Files - This will delete all data files in the currently specified data folder.

Batch Mode Parameters:
  • Run in Batch Mode - If checked, batch parameters become enabled and allow the application to process files in a "Batch Mode". If unchecked, individual files or realtime mode may be chosen in the standard settings window.

  • Start Date - Date of the first historical file to be processed.

  • Stop Date - Date of the last historical file to be processed.

  • Tape Set - The NxCore 2 or 3 letter tape set code.

  • Tape Ext - The NxCore tape extension (nxc, nx2 or nx3).

  • Tape Path - Path to the historical tapes.

  • Use Year Subfolders - Many users will organize their historical tapes by year (such as Data\2016, Data\2015, etc). If 'Use Year Subfolders' is enabled then the application will search in the Tape Path folder + the year sub-folder for the specific NxCore historical tape file.

  • Continue on Errors - If checked and an error occurs (such as a missing file in the sequence) processing will continue with the next file. If unchecked processing will stop on the first error encountered.

  • Run in Reverse Direction - When selected, Batch processing will occur from the stop date to the start date. If not selected batch processing occurs from the start date to the stop date.

Other Considerations:
  • Multiple Symbols/File Limitations - When using multiple symbols AND choosing to use symbols in the filename (IE separating output files by symbol), one thing that is important to know is that Windows has a limit of 2048 files that can be open simultaneously. In fact, using the simple wildcard of " eA* " (all equities beginning with the letter 'A'), could surpass that file limit (unless listing exchange filters are set to limit the # of listing exchanges and hence, symbols).

    It is also impractical to open/close individual files for every trade, quote or depth quote received, as this would increase the processing time of files exponentially. As such, all files must be opened simultaneously prior to data being output. When using multiple symbols it is very important to understand this limitation.

    If you do not separate the output by symbols (IE output data for all symbols to a single file), then the sky is the limit as there would be a maximum of 3 files open simultaneously (if trades, quotes and depth quotes are output to separate files) per trading day.

  • Symbol File Format - When reading symbols to use from a file, the file format is Symbol, Linebreak, Symbol, Linebreak, Symbol, Linebreak, etc etc. As an example:

    eIBM
    eAAPL
    eCSCO
    eLLY

  • Appending Multiple Days to a Single File - When running in this mode, files of the same name will not be written to as new files. Instead, data is appended to files that already exist. So for instance if you run a days worth of data, then decide to change the data parameters slightly and re-run the same day (with all other file settings the same), the file that is created will then contain both days worth of data.

    Should you need to make fresh runs of data previously generated (using this mode), you will need to delete the existing files prior to generating new data.

  • Display Output in Activity Monitor - Displaying all the data output to the activity monitor is a great way to debug/monitor the data to be sure it is being output correctly. However, a significant performance hit is taken while doing so, due to the sheer number of messages that may be generated. For fastest throughput of data this option should be disabled.


JTools Table of Contents