The science behind Target’s recommendations algorithms

Item-Based algorithms: Include algorithms that follow the logic “People who viewed/bought this item also viewed/bought these items.” These algorithms are grouped under the umbrella term item-item collaborative filtering, as well as Items with Similar Attributes algorithms.

User-Based algorithms: Include the Recently Viewed and Recommended for You algorithms.

Popularity-Based algorithms: Include algorithms that return the top-viewed or top-purchased items across the website, or top-viewed or top-purchased by category or item attribute.

Cart-Based algorithms: Include multi-item based recommendations with the logic “people who viewed/bought these items, also viewed/bought those items.”

Custom Criteria: Include recommendations based on custom files uploaded to Target.

Input data: Behavioral data, in the form of views and purchases of visitors collected when you implement Target or from Adobe Analytics.

Model training:

• Data cleansing and sampling: For algorithms with an N-day lookback, the behavioral data is first filtered to include only those N days of data. Collection rules and global exclusions are then applied to remove any items that should not be recommended. Finally, any visitors who interacted with more than 1,000 items have their usage data sampled to only 1,000 items.
• Item similarity computation: This is the core computational step: calculating the log likelihood ratio similarity between all candidate item pairs, and ranking pairs of items by this similarity score.
• Offline filtering: Finally, any further applicable dynamic filters are applied (for example, dynamic category exclusions). After this step, pre-computed recommendations are cached globally to be available for serving.

Model serving: Recommendations content is delivered from Target’s global “Edge” network. When mbox requests are made to Target and it is determined that recommendations content should be delivered to the page, the request for the appropriate item key for the recommendations algorithm is either parsed from the request or looked up from the user profile, and then used to retrieve the recommendations computed in the previous steps. Further dynamic filters are applied at this time, before the appropriate design is rendered.

Input data: As described before, this algorithm is based purely on catalog data (ingested to Target via a Catalog Feed, the Entities API, or from on-page updates.

Model training:

• Attribute extraction: After the application of regular static filters, catalog rules and global exclusions, this algorithm extracts relevant textual fields from the entity schema. Target automatically uses the name, message, and category fields from the entity attributes and attempts to extract any string fields from custom entity attributes. This process is done by ensuring that the majority of values for that field are not parsable as a number, date, or boolean.

• Stemming and stop-word removal: For more accurate text similarity matching, it is prudent to remove very common “stop” words that do not significantly alter the meaning of an item (for example, “was,” “is,” “and,” and so forth). Similarly, stemming refers to the process of reducing words with different suffixes to their root word, which has an identical meaning (for example, “connect,” “connecting,” and “connection” all have the same root word: “connect”). Target uses the Snowball stemmer. Target performs automatic language detection first, and can do stop word removal for up to 50 languages and stemming for 18 languages.

• n-gram creation: After the previous steps, each word is treated as a token. The process of combining contiguous sequences of tokens into a single token is referred to as n-gram creation. Target’s algorithms consider up to 2-grams.

• tf-idf computation: The next step involves the creation of tf-idf vectors to reflect the relative importance of tokens in the item description. For each token/term t in an item i, in a catalog D with |D| items, the term frequency TF(t, i) is computed first (the number of times the term appears in the item i), as well as the document frequency DF(t, D). In essence, the number of items where the token t exists. The tf-idf measure is then

  

  Target uses Apache Spark’s tf-idf featurization implementation, which under the hood hashes each token to a space of 218 tokens. In this step, customer-specified attribute boosting and burying is also applied by adjusting the term frequencies in each vector based on settings specified in the criteria.

• Item similarity computation: The final item similarity computation is done using an approximate cosine similarity. For two items, A and B, with vectors tA and tB, the cosine similarity is defined as:

  

  To avoid significant complexity in computing similarities between all N x N items, the tf-idf vector is truncated to contain only its largest 500 entries, and then compute cosine similarities between items using this truncated vector representation. This approach proves to be more robust for sparse vector similarity computations, as compared to other approximate nearest neighbor (ANN) techniques, such as locality sensitive hashing.

• Model serving: This process is identical to item-item collaborative filtering techniques described in the previous section.

Input data: This is identical to item-item collaborative filtering (CF) methods. Both Recommended For You and Cart-Based algorithms use behavioral data, in the form of views and purchases of users collected when you implement Target or from Adobe Analytics.

Model training:

• Data cleansing and sampling: This is again the same as for collaborative filtering methods, where the lookback window is applied to filter behavioral data to an appropriate date range, followed by application of catalog rules and global exclusions. Visitors who have interacted with more than 1,000 items have only their most recent 1,000 usages considered.
• Train test split: Perform a chronological split of the usages for each user, allocating the first 80% of their usages to training data, with the remaining 20% allocated to the test data.
• Item similarity model training: The core item similarity computation differs for Recommended For You and Cart-Based algorithms in the way that candidate item vectors are constructed. For Recommended For You, the item vectors have dimension NUsers, where each entry represents the sum of implicit ratings for that user of the item–purchases of an item are given a weight of 2x that of views of the item. For Cart-Based recommendations, the item vectors have binary entries; if within-session behavior is to be considered only, there is a new entry for every session. Otherwise, there is an entry in this item vector for every visitor.

The training step computes several types of vector similarities: LLR similarity (discussed here), cosine similarity (defined previously), and a normalized L2 similarity, defined as:

• Item similarity model evaluation: The model evaluation is done by taking the recommendations generated in the previous step and making predictions on the test data set. The online scoring phase is mimicked by chronologically ordering each user’s item usages in the test dataset, then making 100 recommendations for ordered subsets of items in an attempt to predict subsequent views and purchases. An information retrieval metric, the [Mean Average Precision](https://en.wikipedia.org/wiki/Evaluation_measures_(information_retrieval?lang=en), is used to evaluate the quality of these recommendations. This metric takes into account the order of recommendations, and favors relevant items higher up in the list of recommendations, which is an important property for ranking systems.
• Model selection: After offline evaluation, the model that has the highest Mean Average Precision is selected, and all individual item-item recommendations computed for it.
• Offline filtering: The final stage of model training is the application of any applicable dynamic filters. After this step, pre-computed recommendations are cached globally to be available for serving.

Model serving: Unlike previous algorithms in which serving recommendations involve specifying a single key for retrieval, followed by application of business rules, the Recommended for You and Cart-Based algorithms employ a more complex runtime process.

• Multi-key retrieval and merging: For Cart-Based recommendations, up to ten items that are passed in the cart are considered as keys for retrieval and recommendations from each are weighted equally. For Recommended for You, up to the last five unique viewed items and last five unique purchased items are considered as keys for retrieval, with recommendations arising from purchased items weighted twice as much as recommendations arising from viewed items. When merging recommendations, if an item appears in multiple individual lists of recommendations, its weighted similarity scores are added. The final list of recommendations from this stage is then the merged list of re-weighted recommendations, ranked in descending order.
• Filtering: Next, filtering rules such as removal of previously viewed and/or purchased items, as well as other dynamic business rules are applied.