Information about the fields of data that the data workbench server can process to construct a dataset.
The event data used to build a dataset resides in files referred to as log sources. The data available in the log sources is called event data because each data record represents a transaction record or a single instance of an event with an associated timestamp.
A log source’s event data is collected in real-time by Sensors. Event data collected by Sensors from HTTP and application servers is transmitted to data workbench servers, which convert the data into compressed log ( .vsl) files. Event data that resides in a flat file, XML file, or an ODBC data source is read by the data workbench server, which provides decoders that you define to extract a common set of data fields from these different formats.
The following sections provide information about the data fields (referred to as event data record fields or log entry fields ) that are collected by Sensors or read and made available to the data workbench server.
The names of the fields generally follow the naming convention for the W3C extended log file format. Many of the fields have prefixes that indicate the source of the information contained in the field:
Log ( .vsl) files contain the fields of event data that are collected from servers by Sensors and used by the data workbench server in the dataset construction process. The following table lists the fields in a typical event data record as recorded by Sensor:
The IP address of the client as included in the request made to the server.
The cookies sent by the client with the request.
Example: v1st=42FDF66DE610CF36; ASPSESSIONIDQCATDAQC=GPIBKEIBFBFIPLOJMKCAAEPM;
The HTTP referrer string sent by the client to the server with the request.
The string sent by the client with its request to the server that indicates what type of user agent the client is.
Example: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7) Gecko/20040707 Firefox/0.9.2
The method type of the HTTP request.
The query string portion of URI (stem + query string = URI). This is preceded by a question mark (?) and may contain one or more name-value pairs separated by ampersands (&).
The stem portion of URI (stem + query string = URI). The stem is the actual or logical path to the requested resource on the server.
The content type of the resource being requested by the client as reported by the server.
Examples: text/html, image/png, image/gif, video/mpeg
The number of bytes of data sent from the server to the client in response to the request
The status code returned to the client by the server.
The fully qualified domain name or IP address of the host of the requested resource.
The list of all the controlled experiment names and groups that the client is a member of at the time of the request.
The date and time (GMT) at which the request was received by the server. The time is expressed as the number of 100 nanoseconds since January 1, 1600.
Example: 127710989320000000 would be the x-timestamp value for 11:28:52.0000000 on Tuesday, September 13, 2005.
The 64-bit, hexadecimal value of the unique browser identifier found in a persistent cookie as set by a Sensor and provided by the client with a request to a server.
The table below lists examples of fields that are derived by the data workbench server from the baseline event data record fields:
|cs(cookie)(name)||The value of a given name-value pair within a cookie.|
The domain name or IP address of the HTTP referring URI.
Note: This field is read-only.
The entire hostname of the referrer.
Example: If cs(referrer) is https://my.domain.com/my/page , cs(referrer-host) is my.domain.com .
The value of a referrer query string.
Note: You cannot access a referrer query string value using the cs(referrer)(name) field.
The complete URI (stem + query string = entire URI).
The value associated with the given name. If multiple values exist for the given name, this field returns the last of those values.Examples:
|ctime||x-timestamp expressed as seconds since January 1, 1970. This field is also called x-unixtime.|
|date||x-timestamp in the format YYYY-MM-DD.|
|time||x-timestamp in the format HH:MM:SS.|
x-timestamp converted to the local timezone that is specified in the Transformation.cfg file for the dataset. The format is YYYY-MM-DD HH:MM:SS.mmm.
Note: You also can define time conversions such as x-local-timestring in the Log Processing.cfg file. For information, see Log Processing Configuration File .
The identifier corresponding to the log source for a particular log entry. For the identifier to be recorded, you must specify it in the Log Source ID field of the Log Processing.cfg file when defining Sensor , log file, or ODBC data sources. For more information, see Log Processing Configuration File .
Example: from VSensor01.
|x-mask||The mask pattern of the Sensor data sources (derived from the .vsl file names). For a file whose name is of the format YYYYMMDD-SENSORID.VSL , x-mask is SENSORID.|
|x-timestring||x-timestamp in the format YYYY-MM-DD HH:MM:SS.mmm.|
|x-unixtime||The decimal UNIX time derived from x-timestamp.|
Sensor, when used on a server, can collect fields of event data from any valid HTTP request or response header or variable available to it through the server’s API. To collect such fields of data, you must specify the desired header fields or variables in the txlogd.confconfiguration file for Sensor. For more information, see the Data Workbench Sensor Guide.