Defining Exporters

IMPORTANT

Read more about Data Workbench’s End-of-life announcement.

Exporters provide the instructions for outputting the event data.

Transformation functionality provides three types of exporters for exporting .vsl files, log files, XML files, and ODBC data as .vsl files, text files, or delimited text files that can be used by DataWarehouse loading routines, auditing agencies, or other targets.

NOTE

For an exporter to work properly, the log source must meet the appropriate requirements discussed in the Log Sources section of Log Processing Configuration File.

To define an exporter

  1. Open Transform.cfg in data workbench. See To edit the Insight Transform.cfg file.

  2. Right-click Exporters, then click Add New.

  3. Select one of the following options:

    • ExportTextFile
    • ExportDelimitedTextFile
    • ExportVSLFile
    NOTE

    For the ExportVSLFile option, all of the extended fields in the input file and all user-defined fields of the form cs(header) are always written to the VSL output file. If you overwrite an existing extended field, the new value is written to the output file, even if the field is blank.

  4. Edit the Exporters parameters in the configuration file using the following table as a guide:

    Parameter Description
    Data Format

    For ExportTextFile only. The format of each output line, consisting of field name escapes (expressed as %fieldname%) and any other desired fixed text. The format should include a line separator, typically [CR] [LF].

    A literal percent sign (%) can be embedded in the format string by escaping the character as shown here: %%

    An example of an entry for the Data Format parameter is %x-timestring% %x-trackingid%[CR][LF].

    Fields For ExportDelimitedTextFile only. Names of the fields to be output.
    Delimiter

    Optional. For ExportDelimitedTextFile only. Character that is used to separate the fields in the output file.

    The software can not escape delimiters that are included in the data's values. As a result, Adobe does not recommend using commas as delimiters.

    If you hold down the Ctrl key and right-click within the Delimiter parameter, an Insert menu appears. This menu contains a list of special characters that often are used as delimiters.

    Line Separator Optional. For ExportDelimitedTextFile only. The character(s) used to separate lines in the output files. The default value is [CR] [LF].
    Name

    Optional. Identifier for the exporter. This name appears in the Detailed Status interface.

    For information about the Detailed Status interface, see the Data Workbench User Guide.

    Comments Optional. Notes about the exporter.
    Output Path

    Path where output files are to be stored. The path is relative to the data workbench server installation folder.

    Note: The data workbench server that stores output data is processing server #0 in the profile.cfg file.

    File Rotation Period

    Optional. The frequency at which data is exported to the output file. Each output file contains data related to a specific time period called the rotation period. All time calculations are in GMT: One day starts at midnight GMT and ends the following day at midnight GMT, even if the data written to the file includes a field which has been transformed to local time.

    Available values are as follows:

    • YEAR. Each file contains data for one calendar year.
    • MONTH. Each file contains data for one calendar month. Months are numbered 1 (January) through 12 (December).
    • WEEK. Each file contains data for one week. A week starts on Monday. The week starting on one of the first seven days of the year is week 1, and the prior (partial) week, if any, is week 0.
    • DAY. Each files contains data for one calendar day.
    • HOUR. Each file contains data for one hour.
    • NONE. No rotation is performed. All of the data is written to the same file (or a set of files as determined by other parameter settings). See the File Name Format parameter in this table.

    The default file rotation period is DAY.

    • Set the file rotation to NONE only when working in Offline Mode. See the Offline Mode parameter description.
    File Name Format

    Optional. The format of the output file name.

    Each log entry can be stored in a file whose name is derived from the start time of the rotation period, and optionally, from values of fields in the rows that it contains. The fields to use in the file name are embedded as field name escapes (expressed as %fieldname%).

    The file name components related to the rotation period are embedded in the format string using the following escape sequences:

    • %yyyy% (four-digit year)
    • %yy% (two-digit year)
    • %mm% (two-digit month, 01 - 12)
    • %ww% (two-digit week, 01 - 52)
    • %dd% (two-digit day, 01 - 31)
    • %HH% (two-digit hour, 00 - 23)

    The default file name format is %yyyy%%mm%%dd%-%x-mask%.txt

    • The escape sequences are case-sensitive.
    • When File Rotation Period is set to NONE, an empty string is substituted for each of the escape sequences, if present.
    • An error is generated if File Name Format does not result in a unique file name for each rotation period (see the File Rotation Period parameter in this table). For example, when using the DAY rotation period, the %dd%, %mm%, and %yy% or %yyyy% escape sequences must be present in the pattern to avoid data loss.
    • If you are using field name escape sequences within the pattern and the given field has many distinct values, many output files are written for each rotation period. Note that this scenario may result in poor performance, so you should use this feature with caution.
    • All times calculations are in GMT.
    Execute at Rollover

    Optional. When each file is finalized, an external (Windows) command can be executed. The command line is derived from the final file name by substituting the following escape sequences into this parameter:

    • %dir%. The directory part of the final file name, including the trailing backslash.
    • %file%. The file name (excluding the directory and extension) of the final file.
    • %ext%. The extension (including the leading ".") of the final file name.
    • %path%. The full path name of the file, equivalent to %dir%%file%%ext%.

    By default, this parameter is empty (no command is executed).

    Memory Limit

    Optional. The amount of memory in bytes used for buffering the exporter's output. The default value is 10,000,000 bytes.

    Note: If you have many output files that are open at the same time, you may want to increase this value, but you may decrease the amount of memory available for use by other components of the system. Decreasing this value, however, may slow down the export process. For assistance, contact Adobe.

    Open Files Limit

    Optional. The maximum number of files that may be open at the same time for output from the exporter. If this number is exceeded, an error is recorded in the event log and the data workbench server stops running. The default value is 1000.

  5. After you have defined your exporter (and made changes to other parameters) in the Transform.cfg file, save the file locally and save it to the appropriate profile on the data workbench server machine.

On this page