CSV Input Format Parameters

Log Parser

CSV Input Format Parameters

The CSV input format supports the following parameters:

headerRow
  Values: ON | OFF
  Default: ON
  Description: Specifies whether or not the input CSV file(s) begin with a header line.
  Details: When this parameter is set to "ON", the CSV input format assumes that each file being parsed begins with a header line, containing the labels of the fields in the file. If the "iHeaderFile" parameter is left unspecified, the CSV input format will use the field names in the first file's header as the names of the input record fields. If a value is specified for the "iHeaderFile" parameter, the CSV input format will ignore the header line in each file being parsed.
When this parameter is set to "OFF", the CSV input format assumes that the file(s) being parsed do not contain a header, and parses their first line as data records.
For more information on headers and field names, see CSV Input Format Fields.
  Example: -headerRow:OFF
 
iHeaderFile
  Values: path to a CSV file
  Default: not specified
  Description: File containing field names.
  Details: When parsing CSV files that do not contain a header line, the fields of the input records produced by the CSV input format are named "Field1", "Field2", ...
To override this behavior and use meaningful field names, this parameter can be set to to the path of a CSV file containing a header line, causing the CSV input format to use the field names in the specified CSV file's header line as the names of the input record fields. Only the first line of the specified CSV file is parsed, and eventual additional lines are ignored.
For more information on headers and field names, see CSV Input Format Fields.
  Example: -iHeaderFile:"C:\My Folder\header.csv"
 
fixedFields
  Values: ON | OFF
  Default: ON
  Description: Specifies whether or not all the records in the input CSV file(s) have a fixed number of fields.
  Details: When this parameter is set to "ON", the CSV input format assumes that the number of fields in all the input CSV records equals the number of fields found in the first CSV line parsed, or the number of fields specified for the "nFields" parameter.
When this parameter is set to "OFF", the CSV input format assumes that the input CSV records have a variable number of fields, and it parses the first n lines of the input CSV data to determine the maximum number of fields in the records, where n is the value specified for the "dtLines" parameter.
For more information on how the number of fields is determined, see CSV Input Format Fields.
  Example: -fixedFields:OFF
 
nFields
  Values: number of fields (number)
  Default: -1
  Description: Number of fields in the CSV data records.
  Details: When the "fixedFields" parameter is set to "ON", this parameter specifies the number of fields in the input CSV data.
When the "fixedFields" parameter is set to "OFF", this parameter specifies the minimum number of fields in the input CSV data. If the first n lines of input data contain more fields than the specified number of fields, where n is the value of the "dtLines" parameter, then the number of fields is assumed to be the maximum number of fields found within the n lines of data.
The special "-1" value specifies that the number of fields is to be deducted by inspecting the first n lines of input data, where n is the value of the "dtLines" parameter.
For more information on how the number of fields is determined, see CSV Input Format Fields.
  Example: -nFields:3
 
dtLines
  Values: number of lines (number)
  Default: 10
  Description: Number of lines examined to determine number of fields and field types at run time.
  Details: This parameter specifies the number of initial lines that the CSV input format examines to determine the number of the input record fields and the data type of each field.
If the value is 0, all fields will be assumed to be of the STRING data type.
For more information on how the number of fields and their data types are determined, see CSV Input Format Fields.
  Example: -dtLines:50
 
iDQuotes
  Values: Auto | Ignore
  Default: Auto
  Description: Behavior with double-quoted fields.
  Details: When this parameter is set to "Auto" and a field value is enclosed within double-quote characters ("), the CSV input format parses the field ignoring comma characters (,) within the double-quotes, and returns the enclosed value stripping off the surrounding double-quote characters.
When set to "Ignore", the CSV input format does not perform any double-quote processing, and field values are returned verbatim, including double-quote characters.
  Example: -iDQuotes:Ignore
 
nSkipLines
  Values: number of lines (number)
  Default: 0
  Description: Number of initial lines to skip.
  Details: When this parameter is set to a value greater than zero, the CSV input format skips the first n lines of each input file before parsing its header line, where n is the value specified for this parameter.
  Example: -nSkipLines:5
 
comment
  Values: any string
  Default: not specified
  Description: Skip lines beginning with this string.
  Details: When this parameter is set to a non-empty string, the CSV input format skips all the input CSV lines that begin with this string.
  Example: -comment:"Meta Data:"
 
iCodepage
  Values: codepage ID (number)
  Default: 0
  Description: Codepage of the CSV file.
  Details: 0 is the system codepage, -1 is UNICODE.
  Example: -iCodepage:1245
 
iTsFormat
  Values: timestamp format
  Default: yyyy-MM-dd hh:mm:ss
  Description: Format of timestamp values in the input CSV data.
  Details: This parameter specifies the date and/or time format used in the CSV data being parsed. Values of fields matching the specified format are returned as values of the TIMESTAMP data type. For more information on date and time formats, see Timestamp Format Specifiers.
  Example: -iTsFormat:"MMM dd, yyyy"
 
iCheckpoint
  Values: checkpoint filename
  Default: not specified
  Description: Load and save checkpoint information to this file.
  Details: This parameter enables the "Incremental Parsing" feature that allows sequential executions of the same query to only process new events that have been logged since the last execution. For more information, see Parsing Input Incrementally.
  Example: -iCheckpoint:C:\Temp\myCheckpoint.lpc
 


© 2004 Microsoft Corporation. All rights reserved.