CSV Input Format Parameters
The CSV input format supports the following parameters:
| headerRow | ||
| Values: | ON | OFF | |
| Default: | ON | |
| Description: | Specifies whether or not the input CSV file(s) begin with a header line. | |
| Details: | When this parameter is set to "ON", the CSV
input format assumes that each file being parsed begins with a header line, containing
the labels of the fields in the file. If the "iHeaderFile" parameter is left
unspecified, the CSV input format will use the field names in the first file's header as the
names of the input record fields. If a value is specified for the "iHeaderFile"
parameter, the CSV input format will ignore the header line in each file being parsed. When this parameter is set to "OFF", the CSV input format assumes that the file(s) being parsed do not contain a header, and parses their first line as data records. For more information on headers and field names, see CSV Input Format Fields. |
|
| Example: | -headerRow:OFF | |
| iHeaderFile | ||
| Values: | path to a CSV file | |
| Default: | not specified | |
| Description: | File containing field names. | |
| Details: | When parsing CSV files that do not contain a header line,
the fields of the input records produced by the CSV input format are named "Field1",
"Field2", ... To override this behavior and use meaningful field names, this parameter can be set to to the path of a CSV file containing a header line, causing the CSV input format to use the field names in the specified CSV file's header line as the names of the input record fields. Only the first line of the specified CSV file is parsed, and eventual additional lines are ignored. For more information on headers and field names, see CSV Input Format Fields. |
|
| Example: | -iHeaderFile:"C:\My Folder\header.csv" | |
| fixedFields | ||
| Values: | ON | OFF | |
| Default: | ON | |
| Description: | Specifies whether or not all the records in the input CSV file(s) have a fixed number of fields. | |
| Details: | When this parameter is set to "ON", the CSV input
format assumes that the number of fields in all the input CSV records equals the number
of fields found in the first CSV line parsed, or the number of fields specified for the
"nFields" parameter. When this parameter is set to "OFF", the CSV input format assumes that the input CSV records have a variable number of fields, and it parses the first n lines of the input CSV data to determine the maximum number of fields in the records, where n is the value specified for the "dtLines" parameter. For more information on how the number of fields is determined, see CSV Input Format Fields. |
|
| Example: | -fixedFields:OFF | |
| nFields | ||
| Values: | number of fields (number) | |
| Default: | -1 | |
| Description: | Number of fields in the CSV data records. | |
| Details: | When the "fixedFields" parameter is set to
"ON", this parameter specifies the number of fields in the input CSV data. When the "fixedFields" parameter is set to "OFF", this parameter specifies the minimum number of fields in the input CSV data. If the first n lines of input data contain more fields than the specified number of fields, where n is the value of the "dtLines" parameter, then the number of fields is assumed to be the maximum number of fields found within the n lines of data. The special "-1" value specifies that the number of fields is to be deducted by inspecting the first n lines of input data, where n is the value of the "dtLines" parameter. For more information on how the number of fields is determined, see CSV Input Format Fields. |
|
| Example: | -nFields:3 | |
| dtLines | ||
| Values: | number of lines (number) | |
| Default: | 10 | |
| Description: | Number of lines examined to determine number of fields and field types at run time. | |
| Details: | This parameter specifies the number of initial lines
that the CSV input format examines to determine the number of the input record fields and
the data type of each field. If the value is 0, all fields will be assumed to be of the STRING data type. For more information on how the number of fields and their data types are determined, see CSV Input Format Fields. |
|
| Example: | -dtLines:50 | |
| iDQuotes | ||
| Values: | Auto | Ignore | |
| Default: | Auto | |
| Description: | Behavior with double-quoted fields. | |
| Details: | When this parameter is set to "Auto" and a field
value is enclosed within double-quote characters ("), the CSV input format parses
the field ignoring comma characters (,) within the double-quotes,
and returns the enclosed value stripping off the surrounding double-quote characters. When set to "Ignore", the CSV input format does not perform any double-quote processing, and field values are returned verbatim, including double-quote characters. |
|
| Example: | -iDQuotes:Ignore | |
| nSkipLines | ||
| Values: | number of lines (number) | |
| Default: | 0 | |
| Description: | Number of initial lines to skip. | |
| Details: | When this parameter is set to a value greater than zero, the CSV input format skips the first n lines of each input file before parsing its header line, where n is the value specified for this parameter. | |
| Example: | -nSkipLines:5 | |
| comment | ||
| Values: | any string | |
| Default: | not specified | |
| Description: | Skip lines beginning with this string. | |
| Details: | When this parameter is set to a non-empty string, the CSV input format skips all the input CSV lines that begin with this string. | |
| Example: | -comment:"Meta Data:" | |
| iCodepage | ||
| Values: | codepage ID (number) | |
| Default: | 0 | |
| Description: | Codepage of the CSV file. | |
| Details: | 0 is the system codepage, -1 is UNICODE. | |
| Example: | -iCodepage:1245 | |
| iTsFormat | ||
| Values: | timestamp format | |
| Default: | yyyy-MM-dd hh:mm:ss | |
| Description: | Format of timestamp values in the input CSV data. | |
| Details: | This parameter specifies the date and/or time format used in the CSV data being parsed. Values of fields matching the specified format are returned as values of the TIMESTAMP data type. For more information on date and time formats, see Timestamp Format Specifiers. | |
| Example: | -iTsFormat:"MMM dd, yyyy" | |
| iCheckpoint | ||
| Values: | checkpoint filename | |
| Default: | not specified | |
| Description: | Load and save checkpoint information to this file. | |
| Details: | This parameter enables the "Incremental Parsing" feature that allows sequential executions of the same query to only process new events that have been logged since the last execution. For more information, see Parsing Input Incrementally. | |
| Example: | -iCheckpoint:C:\Temp\myCheckpoint.lpc | |