JTOOLS INTERVAL FILE GENERATOR - STOCKS/INDEXES/OPTIONS SPECIFIC JTools Table of Contents Project Location: \JTools_IntervalFileGenerator_Stocks The Interval File Generator for Stocks and Stock Options is very similar to the Standard Interval File Generator but contains some modifications and special features. Like the Standard Interval Generator, the data is true interval data. That is to say that if the timing interval is set to 1 minute then the values generated are for 1 specific minute in time. The JTools_IntervalFileGenerator for Stocks/Stock Options allows you to dump trade and nbbo information in almost any fashion you desire. The system gathers information from NxCore Trade/Quote messages for interval data. The application can be run from state files generated just before or at the desired time. Fields generated and field order are determined by the usage strings (see below for a complete list of possible fields). It will dump the output to the console (if specified to) and to a dated text file(s). Data is output for each day to one file per symbol, or to a data-time series file containing all symbols (unlike the Standard Interval Generator which outputs data for all symbols only into date-time series files). Data is also collected using a different method than the Standard Interval Generator, in that each symbol has it's own composite structure with X possible data points (allocated at realtime and dependant on the timeframe requested). This is possible as the symbol scope (and hence memory usage) is more limited than the Standard Interval Generator (which can contain all symbols for all data types, equities, futures, indexes, etc). Another feature not possible with the standard interval generator is the ability to correct bar data based on cancellations and trade corrections by capturing Category 25 Bar Correction Messages. The JTools Interval Generator for Stocks/Stock Options also combines the functionality of the JTools Batch Processor to make processing data which spans days, months or years trivial. Interval data timestamps will mark the beginning of the interval. For instance, if 5 minute interval data is chosen, an interval at 9:30 will contain data from 9:30:00 through 9:34:59. The Open price reflects the first price recorded for the specific interval. The last price will represent the last price recorded for the specific interval. The high and low price reflect the highest price and lowest price during the specific interval. Volume will reflect the total volume accumulated during the specific interval. Request Data Parameters: Read Symbols From File - If checked, symbols to output will be read from the text file specified in Sym File. Sym File - You may read the symbols you are interest in generating data for from a text file. The Sym File designates the file. You may type it in or Browse for the specific file. Read Symbols From File must be enabled. A sample file, "Symbols.txt" is provided in the project folder. Stock Symbol - The symbol(s) to be output (if Read Symbols From File is disabled). The symbol(s) may be entered in one of the following manners: - A single symbol, such as " eAAPL " - Multiple, comma delineated symbols, such as " eAAPL,iSPX,eGOOG,iDJI " - A wildcard symbol, such as " eA* ". Note that if a wildcard is entered it must be the only entry. Generate Options Data - If checked, options for the underlying stocks and/or indexes will also be generated. Listed Exg Filter - "0"=Include all exchanges. If only specific exchanges are desired, enter those exchange here, comma separated. IE for NYSE and NQEX - 3,12 Time and Interval Period Parameters: Start Time (HHMM) - Time that the application will start collecting/generating interval data. Example - "09:30". Stop Time (HHMM) - Time that the application will stop collecting/generating interval data. Example - "16:00". Bar Interval (1-60) - Amount of minutes each interval represents. Data Field Parameters: Data Usage/Output String - Possible values: (s) Symbol (e) Listed Exg (d) Date (t) Time (1) Open (2) High (3) Low (4) Last (5) NetChg (6) Total Volume (7) Tick Volume (a) Best Ask (b) Best Bid (8) Best Bid Size (9) Best Ask Size (v) VWAP (O) Open Interest (U) Underlying Symbol (/) Blank Field. The field order of the output data will match the field order of the Data Usage String supplied. Misc Parameters: Print Headers - Print headers in output file if checked. Exg Codes as Text - If checked, exchange codes are output as text, otherwise they are output as numeric. Process Empties - If checked issues with no data for the interval will be included and zeroed. If unchecked, only intervals where trading data was recorded will be output. Carry Through OHLC on Empty- If checked, a bar that has no trade data will carry the previous bar values for the Open,High,Low,Last values. Carry Through Bid Ask on Empty - If checked, a bar that has no new quote data will carry the previous bar values for the Best Bid, Best Ask, Best Bid Size and Best Ask Size. Output Symbol Prefix - If checked symbol prefixes will be output with the symbol. Decimal precision - ("0" - "5", "-1" for auto based on price type). Price Filter Level - NxCore halt built in analysis on every trade that can help determine possible erroneous pricing (which may cause spikes when charting the data). One such item is the Filter Level field within the NxCTAnalysis structure. You may see the full documentation for this HERE. For example, if you wish to for eliminate all trades with a filter level of 6 or higher, set the Price Filter Level to 6. Note that the volume for these trades will still be included in the total interval volume, but the pricing will be ignored. Process Saturday - If checked files will be generated on Saturday. Process Sunday - If checked files will be generated on Sunday. Odd Lots Can Set Price - Technically, Odd Lot trades are not eligible to set the last sale price (although volume is considered in official total volume) and therefore do not set any pricing fields in the interval data. However, in some equities it may be advantageous to include Odd Lot pricing, esp. in illiquid stocks where trading is thin and odd lots may in fact be the only trades in a specific interval. Enable this option if you wish to have odd lots included in the interval pricing. Odd Lots Can Set Volume - Technically, volume from Odd Lot trades are included in the current official total volume. (although are not eligible to set the last sale price). Some of our users however desire to have Odd Lot volume excluded from the interval volumes generated. Disable this option if you wish to have odd lot volume excluded from interval volume. Output File Parameters: Write On Interval - If checked, data files will be generated when each time interval is passed. No corrections are available for data generated in this manner. May be used in conjunction with Write On End-Of-Tape. Write On End-Of-Tape (or Stop Time) - If checked, data files will be generated either a) when the specified Stop Time has been reached or b) at the end of the tape (see below). May be used in conjunction with Write On Interval. Continue Until Complete Msg - If checked and Write On End Of Tape is also specified, data will be written when the tape completes as opposed to the specified Stop Time. This allows any corrections to be processed as corrections do not occur until after the actual trade data, sometimes much later. Symbol Based Files - If checked, files will be named and output according to the stock symbol. If unchecked, all symbols will be named and output to a single date-time based file. Append - One File Per Symbol - If checked, data will be written to one file per symbol, continuously. Symbol Based Files and Write On End-Of-Tape options must also be enabled (checked on). Note: If appending to one file per symbol, the files are never overwritten with new data, but the existing data is appended to. Therefore, should you change parameters in the application and wish to regenerate the data from scratch, you must first Delete the existing files. Use the Delete Files button to accomplish this, or manually delete them through Windows Explorer. Default File Extension - The default file extension to use for filenames. Field Separator - A single character used to separate data fields within the files. Data Folder - Folder data files are created in. If left blank, the output files will be generated in the sub-folder JTools_OutData\IntervalData\Stocks. The sub-folder will be located in the same folder that the application runs from. Delete Files - Deletes all files in the Data Folder with filenames ending with ".IFGSIO.<Default File Extension>" . You will be asked when pressing this button if you are sure, before the deletion occurs. Batch Mode Parameters: Run in Batch Mode - If checked, batch parameters become enabled and allow the application to process files in a "Batch Mode". If unchecked, individual files or realtime mode may be chosen in the standard settings window. Start Date- Date of the first historical file to be processed. Stop Date- Date of the last historical file to be processed. Tape Set- The NxCore 2 or 3 letter tape set code. Tape Ext- The NxCore tape extension (nxc, nx2 or nx3). Tape Path- Path to the historical tapes. Use Year Subfolders - Many users will organize their historical tapes by year (such as Data\2016, Data\2015, etc). If 'Use Year Subfolders' is enabled then the application will search in the Tape Path folder + the year sub-folder for the specific NxCore historical tape file. Continue on Errors - If checked and an error occurs (such as a missing file in the sequence) processing will continue with the next file. If unchecked processing will stop on the first error encountered. Run in Reverse Direction - When selected, Batch processing will occur from the stop date to the start date. If not selected batch processing occurs from the start date to the stop date. Output Data to Activity Monitor - Output of the actual interval data will be displayed in the activity monitor if checked. While this can be useful to determine if data is actually being generated, it can severely impede performance of the application. Output File Status to Activity Monitor - Output of the file write status will be displayed in the activity monitor if checked. Note any file errors will be reported regardless of this setting. Project Location: \JTools_IntervalFileGenerator_Stocks JTools Table of Contents