Hash collisions

Dimensions in Adobe Analytics collect string values. Sometimes these strings are hundreds of characters long, while other times they are short. To improve performance, these string values are not used directly in report-time processing. Instead, a hash is computed for each value, producing a uniform-size identifier. For most fields, the value is converted to lowercase before hashing, which reduces the total number of unique values. All reports run on these hashed values, which drastically increases their performance.

Adobe Analytics maintains a separate hash table for each variable, and each table is rebuilt every month. Within any one of those tables, two different source values can occasionally produce the same hash, known as a hash collision.

Hash collisions can manifest in reports as follows:

Odds of a hash collision

Adobe Analytics uses 32-bit hashes for most dimensions, which means that there are 2³² possible hash combinations (approximately 4.3 billion). The approximate odds of encountering a hash collision based on the number of unique values are as follows. These odds are based on a single dimension for a single month.

Similar to the birthday paradox, the likelihood of hash collisions drastically increases as the number of unique values increases. At 1 million unique values, it is likely that there are at least 100 hash collisions for that dimension.

Mitigating hash collisions

Hash collisions cannot be eliminated entirely, but their impact on reports can be mitigated. Most hash collisions happen with two uncommon values, which have no meaningful impact on reports. Even if a hash collides with a common and uncommon value, the result is negligible. However, in rare cases where two popular values experience a hash collision, it is possible to see its effect clearly. Adobe recommends the following to reduce its effect in reports: