Acquiring Page Request Data

Last update: 2022-10-04
  • Created for:
  • User
    Admin
IMPORTANT

Read more about Data Workbench’s End-of-life announcement.

Sensor acquires all measurement data that is carried on the page requests (GET requests) made to the web servers on which it has been installed.

Sensor acquires this measurement data through the web server’s application programming interface, directly from the instance or instances of the web server software running on your web server. Sensor does not access the web server generated log files. In fact, after Sensor and the data workbench server have been installed and tested, the web server’s native logging feature can be disabled without affecting data collection. In many cases, disabling the logging of files to the local disks of the web server machines themselves improves the page serving capacity of those web servers because of the relatively large amount of fixed disk I/O required to log this information to the local disk of the web server machine.

Sensor collects measurement and web request data directly from each web server process and virtual web server process (if applicable) and temporarily writes the data to a Queue File, a fault-tolerant memory queue with fixed disk backing, on the web server machine. The Sensor Transmitter service (or daemon depending upon the platform) retrieves data from the Queue File and then compresses and encrypts it before transmitting it to the data workbench server for long-term storage. With Sensor, data is accumulated on your web server machines in the Queue File only if you are having a network or other problem that prevents its transmission. The Queue File allows for the efficient local storage of hours to days of web request data to protect the data if a network or system fault does not allow the data to be transmitted to the data workbench server in real time.

Sensor collects measurement data from each physical and logical web server process, filters it by content type, compresses it, encrypts it, and streams it to the data workbench server.

The following table contains the fields of log information that are acquired by Sensor for each GET request that is not filtered out based on Sensor’s configuration file:

W3C Name Data Collected Explanation Explanation
x-trackingid Tracking Identifier (unique visitor) Identifier read from a cookie placed in the user’s browser by Sensor on Visitor’s initial request V1st=3C94007B4E01F9C2

Date

Time

Timestamp Time at which request was processed by server (at 100ns precision; accuracy depends on server environment and NTP) 2002-11-21 17:21:45.123
sc(content-Type) Content Type Type of object returned from server text/html
sc-status HTTP Response Status Code Numerical code generated by the server that notes the status of the HTTP server's response 404
cs-uri-stem URI Stem The stem portion of the URI requested by the client pagedir/page.asp
c-ip Client IP IP Address of the requesting client 127.0.0.1
s-dns Server Domain Name Domain name of the web server processing the request www.domain.com
cs(referrer) Referring URL Contents of the HTTP referrer field sent by the client https://www.referringsite.com
cs(user-agent) User Agent Device used to make a request to the HTTP server Mozilla/4.0+(compatible;+MSIE+6.0; +Windows+NT+5.1)
cs(cookie) Client Cookies from Domain Contents of all of the user’s cookies for the site

KL_TC1 1038058778312

KL972x1038058778312282052

KL_PVKL972 0

cs-uri-query Query String The query string portion, if any, of the URI requested by the client PAGENAME=dynamic1&link=3001
Previous page
Next page

On this page