In Adobe Analytics, you have multiple options for removing bot traffic from reporting:
Both standard and custom bot filtering methods are supported in Analytics > Admin > Report Suites > Edit Settings > General > Bot Rules:
|Standard IAB bot rules||Selecting Enable IAB Bot Filtering Rules uses the IAB’s (International Advertising Bureau’s) International Spiders & Bots List to remove bot traffic. Most customers select this option at a minimum.|
|Custom bot rules||You can define and add custom bot rules based on user agents, IP addresses, or IP ranges.|
For more detail, see Bot rules overview.
The websiteBot plug-in allows you to dynamically identify if desktop visitors are bots. You can use this data to drive greater accuracy in all types of reporting, which gives you a better way to measure legitimate site traffic.
This plug-in performs two checks:
For more information, see the Adobe Analytics Implementation Guide.
In addition, since bots are morphing quickly, Adobe offers several other powerful features that, when combined properly and on a regular basis, can help drive the removal of these enemies of data quality. Those features are: Experience Cloud ID Service, Segmentation, Data Warehouse, Customer Attributes, and Virtual Report Suites. Here is an overview of how you can use these tools.
To start, create a new declared ID in the People Core Service. Pass your visitor’s Experience Cloud ID into this new declared ID, which can be done quickly and easily with tags in Adobe Experience Platform. Let’s use the name “ECID” for the declared ID.
Here is how this ID can be captured via Data Element. Be sure to populate your Experience Cloud OrgID into the Data Element correctly.
Once this Data Element is set up, follow these instructions to pass declared IDs into the ECID Tool using tags in Adobe Experience Platform.
Now that you have your visitor’s ECID passed into a declared ID, you can use segmentation in Analysis Workspace to identify visitors who are acting like bots. Bots are often defined by their behavior: single access visits, unusual user agents, unknown device/browser information, no referrers, new visitors, unusual landing pages, etc. Use the powers of Workspace drill-downs and segmentation to identify the bots that have evaded IAB filtering and your report suite bot rules. For example, here’s a screenshot of a segment that you could use:
Now that you have identified the bots using segments, the next step is to use Data Warehouse to extract all the Experience Cloud IDs associated with this segment. This screenshot shows how you should set up your Data Warehouse request:
Remember to use Experience Cloud Visitor ID as your dimension and apply the ‘Bots’ segment.
Once the Data Warehouse report arrives, you have a list of ECIDs that must be filtered from historical data. Copy and paste these ECIDs into a blank .CSV file with just two columns, ECID and Bot Flag.
Use this .CSV file as your Customer Attribute import file, then subscribe your report suite(s) to the Customer Attribute as described in this blog post.
Once your data set has been processed and integrated into Analysis Workspace, create one more segment that leverages your new “Bot Flag” customer attribute dimension and an Exclude container:
Finally, create a Virtual Report Suite that uses this segment to filter out the identified bots:
This newly segmented Virtual Report Suite will now result in a cleaner set of data, with the identified bots removed.
Set at least a monthly reminder to identify and filter new bots, perhaps before a regularly scheduled analysis.