In Adobe Analytics, you have multiple options for removing bot traffic from reporting:
Both standard and custom bot filtering methods are supported in Analytics > Admin > Report Suites > Edit Settings > General > Bot Rules:
|Standard IAB bot rules||Selecting Enable IAB Bot Filtering Rules uses the IAB’s (International Advertising Bureau’s) International Spiders & Bots List to remove bot traffic. Most customers select this option at a minimum.|
|Custom bot rules||You can define and add custom bot rules based on user agents, IP addresses or IP ranges.|
For more detail, see Bot rules overview.
In addition, since bots are morphing quickly, Adobe offers several other powerful features that, when combined properly and on a regular basis, can help drive the removal of these enemies of data quality. Those features are: Experience Cloud ID Service, Segmentation, Data Warehouse, Customer Attributes, and Virtual Report Suites. Here is an overview of how you can leverage these tools.
To start, you’ll want to create a new declared ID in the People Core Service. You’ll need to pass your visitor’s Experience Cloud ID into this new declared ID, which can be done quickly and easily with Adobe Experience Platform Launch. Let’s use the name “ECID” for the declared ID.
Here is how this ID can be captured via Data Element. Be sure to populated your Experience Cloud OrgID into the Data Element correctly.
Once this Data Element is set up, follow these instructions to pass declared ID’s into the ECID Tool in Launch.
Now that you have your visitor’s ECID passed into a declared ID, you can use segmentation in Analysis Workspace to identify visitors who are acting like bots. Bots are often defined by their behavior: single access visits, unusual user agents, unknown device/browser information, no referrers, new visitors, unusual landing pages, etc. Use the powers of Workspace drill-downs and segmentation to identify the bots that have evaded IAB filtering and your report suite bot rules. For example, here’s a screenshot of a segment that you could use:
Now that you have identified the bots using segments, the next step is to leverage Data Warehouse to extract all the Experience Cloud IDs associated with this segment. This is how you should set up your Data Warehouse request:
Remember to use Experience Cloud Visitor ID as your dimension and apply the Bots segment.
Once the Data Warehouse report arrives, you’ll have a list of ECIDs that need to be filtered from historical data. Copy and paste these ECIDs into a blank .CSV file with just two columns, ECID and Bot Flag.
Use this .CSV file as your Customer Attribute import file, then subscribe your report suite(s) to the Customer Attribute as described in this blog post.
Once your data set has been processed and integrated into Analysis Workspace, create one more segment that leverages your new “Bot Flag” customer attribute dimension and a Exclude container:
Finally, you should create a Virtual Report Suite that leverages this segment to filter out the identified bots:
This newly-segmented Virtual Report Suite will now result in a significantly cleaner set of data, with the identified bots completely removed.
Set at least a monthly reminder to identify and filter new bots, perhaps prior to regularly scheduled analysis.