Data not seen in Full Processing Data Sources

Description

One of the common reasons for the data uploaded via Full Processing Data Sources to be not available in the reports.

Resolution

The small size of the uploaded file is one of the reasons.

Full Processing was designed to simulate the consumption of a continuous stream of data sent in by the client. Each file uploaded represents the next portion of that stream. The hits of each file are grouped together by the visitor, and when the timestamps of those hits clearly indicate that the visit is complete, the data is sent on to final processing and then appears in reporting.

There are two solutions here:

  1. Upload more data: Partially complete visits typically only sit around until the next upload (it’s really based on the timestamps of the hits), so by uploading on a regular basis, visits will appear in reporting on a regular basis. This is how a normal Full Processing account will be used.

  2. Hit the “Complete Processing” link on the Admin Module Data Sources Manager page, or use the API call that does the same thing: This is essentially telling the processor “This is the end of the stream - please consider any visits you’re holding complete and send them along”. Note that hitting this button after every upload will destroy the “simulated stream” - visits that span from one file to the next will be counted as 2 visits instead of one. This option is more useful for testing and shouldn’t be used as part of normal operation.

On this page