Create a data feed

When creating a data feed, you provide Adobe with:

  • The information about the destination where you want raw data files to be sent

  • The data you want to include in each file

  • The frequency of how often the data feed should be sent (including the lookback window if you choose to include late-arriving hits)

Before you create a data feed, it’s important to have a basic understanding of data feeds and to ensure that you meet all prerequisites. For more information, see Data feeds overview.

Create and configure a data feed create-and-configure-data-feed

  1. Log in to experiencecloud.adobe.com using your Adobe ID credentials.

  2. Select the 9-square icon in the upper-right, then select Analytics.

  3. In the top navigation bar, go to Admin > Data feeds.

  4. Select Create data feed.

    A page displays with the following categories: Details, Data formatting, Data structure, Schedule, and Destination.

    New data feed page

  5. In the Details section, complete the following fields:

    table 0-row-2 1-row-2 2-row-2 3-row-2
    Field Function
    Name The name of the data feed. Names must be unique within the selected report suite, and can be up to 255 characters in length. Learn more
    Tags Apply any tags to the data feed for easier categorization. You can filter on tags as described in Filter and search the list of data feeds in Manage data feeds.
    Description Specify a description for the data feed. The description you add is visible when editing the data feed.
  6. In the Data formatting section, specify the following information:

    table 0-row-2 1-row-2 2-row-2 3-row-2 4-row-2 5-row-2 6-row-2 7-row-2 8-row-2
    Field Function
    Compression format The type of compression used. Gzip outputs files in .tar.gz format. Zip outputs files in .zip format.
    Packaging type Select Multiple files for most data feeds. This option paginates your data into uncompressed 2GB chunks. (If the Multiple files option is selected and uncompressed data for the reporting window is less than 2GB, one file is sent.) Selecting Single file outputs the hit_data.tsv file in a single, potentially massive file.
    Manifest

    Choose whether to include a manifest file with each data feed delivery.

    You can choose from the following options:

    • Manifest file: Contains information for each file included in the data feed.
    • Finish file (Legacy): Indicates that the data feed completed successfully. No other information is included. This option is suitable for existing feeds that originally used this option that need to be reprocessed. It is available only when sending data feed data in a single package.
    • None: No file is included
    Send manifest even when no data

    Determines whether Adobe should deliver a manifest file to the destination when no data is collected for a feed interval. If you select Manifest file, you receive a manifest file similar to the following when no data is collected:

    text

    Datafeed-Manifest-Version: 1.0

    Lookup-Files: 0

    Data-Files: 0

    Total-Records: 0

    Replace operating system strings

    When collecting data, some characters (such as new lines) can cause issues. Select this option to have these characters removed from feed files.

    This option detects the following string sequences embedded in customer data and replaces them with a space:

    • Windows: CRLF, CR, or TAB
    • Mac and Linux: \n, \r, or \t
    Enable dynamic lookups

    Dynamic lookups allow you to receive additional lookup files in your data feed that is otherwise not available. This setting enables the following lookup tables to be sent with each data feed file:

    • Carrier name
    • Mobile attributes
    • Operating system type

    For more information, see Dynamic lookups.

    Allow late-arriving hits

    Historical data can arrive after a data feed job finishes processing for a given hour or day, such as through timestamped hits or data sources.

    Select this option to include data that arrived after the data feed job finished processing data within the set reporting frequency (usually daily or hourly). With this option enabled, every time a data feed processes data, it looks at any late hits that arrived and batches them in with the next data feed file that is sent.

    For more information, see Late-arriving hits.

    Lookback window (for late-arriving hits) This option displays when the option Allow late-arriving hits is enabled. Select the lookback window to limit the time frame of late hits that are included. Select Unlimited if you want to allow all late arriving hits, regardless of how late. You can choose a preset interval, such as 1 hour, 2 hours, 1 week, 2 weeks, and so forth. Or, select Custom lookback window, then in the Custom Lookback field specify a lookback window up to 26,280 hours.
  7. In the Data structure section, in the Report suite field, select the source report suite that contains the data that you want to export.

    Consider the following when selecting a report suite:

    • If multiple data feeds are created for the same report suite, each data feed must have different column definitions.
    • Only source report suites support data feeds; virtual report suites are not supported.
    • The list of available columns depends on the login company that the selected report suite belongs to. If you change the report suite, the list of available columns can change.
  8. Use either or both of the following methods to determine which data columns to include in the feed:

    • Add columns individually: In the Available section on the left, select any columns that you want to include, then select Include. All data columns in Adobe Analytics are available. You can select multiple columns by holding Shift, or by holding Command (on macOS) or Ctrl (on Windows). Click Add all to include all columns in a data feed.

      Columns you add appear in the Included section on the right.

    • Add a column template: In the Column templates field, select a column template to add. A column template is a predefined group of columns, and Adobe provides several by default.

      All columns included in the template appear in the Included section on the right.

  9. (Optional) To create a column template that is based on the data feed that you are currently creating, select Save as template, specify a name for the template, then select Save. This option is useful if you plan to create additional data feeds that include the same columns.

    Create column template while creating a data feed

  10. (Optional) To download a list of included columns in .csv format, select Download columns. This option can be useful for data feeds that have a large number of columns.

  11. In the Schedule section, specify the following information:

    table 0-row-2 1-row-2 2-row-2 3-row-2 4-row-2 5-row-2
    Field Function
    Frequency

    Select how often the data feed should be sent. The options that are available are dynamically populated based on your report suite configuration.

    The following options are commonly available:

    • Daily: Feeds contain a full day’s worth of data, from midnight to midnight in the report suite’s time zone. Use this option for backfill or historical data, or for continuing feeds.
    • Hourly: Feeds contain a single hour’s worth of data. Use this option for continuing feeds.

    A 15-minute export frequency is possible, but is not available by default. For this option to become available in your environment, you must first contact Adobe Customer Care and request that your report suite is configured to support 15-minute exports.

    Processing delay Choose whether to wait a given amount of time before processing a data feed file. A delay can be useful to give mobile implementations an opportunity for offline devices to come online and send data. It can also be used to accommodate your organization’s server-side processes in managing previously processed files. In most cases, no delay is needed. You can delay a feed by up to 8 hours (480 minutes), or even longer if you select a custom amount of time (9,999 minutes of delay, or about 1 week).
    Continuous feed When selected, this option removes the end date, allowing a feed to run indefinitely. When a feed finishes processing historical data, a feed waits for data to finish collecting for a given hour or day. When the current hour or day concludes, processing begins after the specified delay.
    Start date Specify the date when you want the data feed to begin. To immediately begin processing data feeds for historical data, set this date to any date in the past when data is being collected. The start date is based on the report suite’s time zone.
    End date Specify the date when you want the data feed to end. The end date is based on the report suite’s time zone.
  12. In the Destination section, configure the destination where you want the data to be sent.

    note note
    NOTE
    Consider the following when configuring a report destination:
    • Adobe recommends using a cloud account for your report destination. Legacy FTP and SFTP accounts are available, but are not recommended.

    • Any cloud accounts that you previously configured are available to use for data feeds. You can configure cloud accounts in any of the following ways:

    • Cloud accounts are associated with your Adobe Analytics user account. Other users cannot use or view cloud accounts that you configure unless you make them available to all users in your organization.

    • You can edit any locations that you create from the Locations manager in Components > Locations

    Complete the following fields:

    table 0-row-2 1-row-2 2-row-2 3-row-2
    Field Function
    Account

    Do either of the following:

    • Use an existing account: Select the drop-down menu next to the Account field. Or, begin typing the account name, then select it from the drop-down menu.

      Accounts are available to you only if you configured them or if they are shared with an organization you are a part of.

    • Create a new account: Select Add new beneath the Account field. For information about how to configure the account, see Configure a location account in Configure cloud import and export accounts.

    Location

    Do either of the following:

    • Use an existing location: Select the drop-down menu next to the Location field. Or, begin typing the location name, then select it from the drop-down menu.
    • Create a new location: Select Add new beneath the Location field. For information about how to configure the location, see Configure a location in Configure cloud import and export locations.
    Notify when complete Specify one or more email addresses where a notification should be delivered after the data feed is successfully sent or fails to send. Multiple email addresses must be separated with a comma.
  13. Select Save.

Manage column templates

Templates allow you to reuse the same columns for future data feeds that you create.

When managing templates, you can create new templates, use templates that are already created, copy templates, edit templates, and delete templates.

Admin > Data feeds > Manage templates

Manage column templates

Create a column template

When creating multiple data feeds that use the same columns, Adobe recommends that you create column templates. Any column templates that you create are available to use by anyone in your organization.

To create a column template:

  1. In Adobe Analytics, go to Admin > Data feeds > Manage templates.

  2. Select Create new template to create a new column template.

    Create column template

  3. In the Template name field, specify a name for the template.

  4. In the Available section on the left, select any columns that you want to include, then select Include. All available data columns in Adobe Analytics are available. You can select multiple columns by holding Shift, or by holding Command (on macOS) or Ctrl (on Windows). Click Add all to include all columns in a data feed.

    Columns you add appear in the Included section on the right.

  5. Select Save.

Edit a column template

  1. In Adobe Analytics, go to Admin > Data feeds > Manage templates.

  2. Select the template that you want to edit, then select Edit.

  3. Make any edits, then select Save.

Copy a column template

  1. In Adobe Analytics, go to Admin > Data feeds > Manage templates.

  2. Select the template that you want to copy, then select Copy.

  3. In the Template name field, specify a name for the template.

  4. Make any additional changes, then select Save.

Delete column templates

  1. In Adobe Analytics, go to Admin > Data feeds > Manage templates.

  2. Select one or more templates that you want to delete, then select Delete.

recommendation-more-help
6b7d49d5-f5fe-4b7f-91ae-5b0745755ed2