Customer Data Feeds customer-data-feeds

Basic information about Customer Data Feed (CDF) files and instructions on how to get started. Start here if you’re interested in receiving CDF files or just want more information.

File Contents and Purpose file-contents-purpose

A CDF file contains the same data that an Audience Manager event call (/event) sends to our servers. This includes data like user IDs, trait IDs, segment IDs, and all the other parameters captured by an event call. Internal Audience Manager systems processes event data into a CDF file with content organized into fields that appear in a set order. Audience Manager tries to generate CDF files hourly and stores them in a secure, customer-specific bucket on an Amazon S3 server. We provide these files so you can work with Audience Manager data outside of the limits imposed by our user interface.

Getting Started getting-started

There is no self-service process to start CDF file delivery. Contact your Audience Manager consultant or Customer Care to get started. During implementation, your Audience Manager representative will:

File notifications and CDF files will appear in your S3 bucket when they’re ready for download. You’re responsible for monitoring and downloading files from your assigned S3 directory. See Customer Data Feed File Processing Notifications.

Next Steps next-steps

The sections below and the Customer Data Feed FAQ can help you become more familiar with this service.

Customer Data Feed Contents Defined cdf-defined

Lists and defines the data elements and arrays in a CDF file, by order of appearance. Definitions include data types, but this information is not part of a CDF file.

Definitions definitions

A CDF file includes some or all of the fields defined below. For information about internal file organization, see Customer Data Feed File Structure.

Customer Data Feed File Structure cdf-file-structure

Lists and defines the data structure of a CDF file. This includes data sequence, field delimiters and separators, a data file map, and sample file.

Data Field Identifiers and Sequence identifiers-and-sequence

CDF files do not contain labeled columns or field headers. Instead, a CDF file defines fields and arrays with non-printing ASCII characters. Also, the CDF file lists each field and array in a specific order. Understanding the field identifiers and order will help you parse the file properly.

CDF File Map cdf-file-map

CDF file data appears in the order shown below.

Identifying Arrays

Arrays in a CDF file start and end with the Ctrl + a field separator. This makes the first element in an array appear like a standalone data field. For example, the realized traits array starts with ^A1234. The array delimiter and ID ^B5678 follows this entry. As a result, you might be tempted to think that the first element in the realized traits array is ID 5678 (because it starts with ^B). This is not the case, which is why you need to be familiar with the sequence and structure of a data file. Even though the first element in the realized trait array (or any of the other arrays in a CDF file) starts with ^A, the order of appearance or position in the file defines the start of an array. And, the first element in an array is always separated from the preceding entry by ^A.

Sample CDF File sample-file

A sample CDF file could look similar to the following. We’ve inserted line breaks into this example to help it fit the page.

Customer Data Feed File Naming Conventions cdf-naming-conventions

The sections below list and define the elements in your CDF file name.

CDF File Name: Syntax and Example cdf-file-name

A typical CDF file name contains the elements listed below. Note, italics indicates a variable placeholder:

Syntax

s3://aam-cdf/YOUR-S3-BUCKET-NAME/day=yyyy-mm-dd/hour=hh/AAM-CDF_PARTNER-ID_FILE-SEQUENCE_0.gz

Example

s3://aam-cdf/dataCompany/day=2017-09-14/hour=17/AAM_CDF_1234_0_0_0.gz

In your S3 storage bucket, files are sorted in ascending order by Partner ID (PID), day, and hour.

CDF File Name Elements Defined cdf-file-name-elements

The following table lists and defines the elements in a CDF file name.

Customer Data Feed File Processing Notifications cdf-file-processing-notifications

Audience Manager writes a .info file to your S3 directory to let you know when your Customer Data File (CDF) is ready for download. The .info file also includes JSON formatted metadata about the contents of your CDF files. Review this section for information about the syntax and fields used by this notification file.

Sample Info File sample-info-file

Each .info file contains a Files and Totals section. The Files section contains an array that holds specific metrics for each hourly file. The Totals section contains metrics aggregated across all your CDF files for a particular day. The contents of your .info file could look similar to the following example.

{
    "Files": [
        {
            "FileByteSize": 2709730,
            "FileChecksumMD5": "a9ea418e79511642cff11c2a898037dc-1",
            "FileName": "AAM_CDF_1109_000000_0.gz",
            "FileSequenceNumber": 1
        },
        {
            "FileByteSize": 2783351,
            "FileChecksumMD5": "7b469485d60274b6991acd0817855840-3",
            "FileName": "AAM_CDF_1109_000001_0.gz",
            "FileSequenceNumber": 2
        }
    ],
    "Totals": {
        "Day": "2017-09-26",
        "Hour": "18",
        "TotalByteSize": 150092997,
        "TotalNumberFiles": 2
    }
}

Info File Fields Defined info-file-fields-defined

The following tables list and define the elements in a CDF .info file.

Files Object

Totals Object

Customer Data Feed File Name Times and File Content Times are Different different-processing-times

Your CDF file contains timestamps in the file name and file contents. These timestamps record different event processes for the same CDF file. It is not uncommon to see different timestamps in the name and contents of the same file. Understanding each timestamp can help you avoid common mistakes when working with this data or trying to sort it by time.

Locating CDF File Timestamps locating-timestamps

CDF files record time differently in 2 separate locations.

Understanding the Difference Between Timestamps understanding-timestamps

The following table provides additional details about your CDF file timestamps along with information about how to use them properly.