Derived fields

Last update: 2024-01-11
  • Created for:
  • Admin

Derived fields are an important aspect of the real-time reporting functionality in Adobe Customer Journey Analytics. A derived field allows you to define (often complex) data manipulations on the fly, through a customizable rule builder. You can then use that derived field as a component (metric or dimension) in Workspace or even further define the derived field as a component in Data view.

Derived fields can save a significant amount of time and effort, compared to transforming or manipulating your data in other locations outside of Customer Journey Analytics. Such as Data Prep, Data Distiller, or within your own Extract Transform Load (ETL) / Extract Load Transform (ELT) processes.

Derived fields are defined within Data views, are based on a set of functions defined as rules, and applied to available standard and/or schema fields.

Example use cases are:

  • Define a derived Page Name field that corrects improper collected page name values to correct page name values.

  • Define a derived Marketing Channel field that determines the proper marketing channel based on one or more conditions (for example URL parameter, page URL, page name).

Derived field interface

When you create or edit a derived field, you use the derived field interface.

Screenshot of the Derived field dialog

Name Description
1 Selector You use the selector area to select and drag and drop your function, function template, schema field, or standard field on to the rule builder.
Use the drop-down to select between:
Function Functions - lists available functions,
Function template icon Function templates - lists available function templates,
Schema field icon Schema fields - lists fields available from dataset categories (event, profile, lookup) and previously defined derived fields, and
Standard field icon Standard fields - standard available fields (like Platform Dataset ID). Only string and numeric standard fields are displayed in the selector. If the function supports other data types, standard fields with these other data types can be selected for values or fields within the rule interface.
You can search for function, function templates, schema, and standard fields using the Search icon Search box.
You can filter the selected object list by selecting Filter icon Filter and specify filters in the Filter fields by dialog. You can easily remove filters using Close icon for each filter.
2 Rule builder You build your derived field sequentially using one or more rules. A rule is a specific implementation of a function and is therefore always associated with only one function. You create a rule by dragging and dropping a function into the rule builder. The function type determines the interface of the rule.
See the Rule interface for more information.
You can insert a function at the start, end, or in between rules already available in the rule builder. The last rule in the rule builder determines the final output of the derived field.
3 Field Settings You can name and describe your derived field and inspect its field type.
4 Final Output This area shows an on-the-fly updated preview of output values, based on data over the last 30 days and the changes you make to the derived field in the rule builder.

Field template wizard

When you access the derived field interface for the first time, the Start with a field template wizard is shown.

  1. Select the template that best describes the type of field you are trying to create.
  2. Select the Select button to continue.

Your derived field dialog is populated with rules (and functions) required or useful for the type of field that you selected. See Function templates for more information on the available templates.

Rule interface

When you define a rule in the rule builder, you use the rule interface.

Screenshot of the Derived Field Rule Interface

Name Description
A Rule Name By default the rule name is Rule X (X referring to a sequence number). To edit the name of a rule, select its name and type in the new name, for example Query Parameter.
B Function Name The selected function name for the rule, for example URL PARSE. When the function is the last in the sequence of functions and determines the final output values, the function name is followed by - FINAL OUTPUT, for example URL PARSE - FINAL OUTPUT.
To show a popup with more information on the function, select Help icon.
C Rule Description You can optionally add a description to a rule.
Select More icon, then select Add Description to add a description or Edit Description to edit an existing description.
Use the editor to enter a description. You can use the toolbar to format the text (using style selector, bold, italic, underline, right, left, centered, color, number list, bullet list) and adding links to external information.
To finish editing the description, click outside of the editor.
D Function Area Defines the logic of the function. The interface depends on the type of function. The dropdown for Field or Value shows all categories of fields (rules, standard fields, fields) available, based on the type of input the function expects. Alternatively, you can drag and drop a field from the Schema and Standard fields selector on to a Field or Value. When that dragged field is originating from a Lookup dataset, a Lookup function is automatically inserted before the function you define.
See Function reference on detailed information for each of the functions supported.

Create a derived field

  1. Select an existing Data view or create a Data view. See Data views for more information.

  2. Select the Components tab of the Data view.

  3. Select Create derived field from the left rail.

  4. To define your derived field, use the Create derived field interface. See Derived field interface.

    To save your new derived field, select Save.

  5. Your new derived field is added to the Derived fields > container, as part of Schema fields in the left rail of your Data view.

Edit a derived field

  1. Select an existing Data view. See Data views for more information.

  2. Select the Components tab of the Data view.

  3. Select Schema fields tab in the Connection pane on the left.

  4. Select Derived fields > container.

  5. Hover over the derived field that you want to edit, and select Edit icon.

  6. To edit your derived field, use the Edit derived field interface. See Derived field interface.

    • Select Save to save your updated derived field.

    • Select Cancel to cancel any changes you made to the derived field.

    • Select Save As to save the derived field as a new derived field. The new derived field has the same name as the original edited derived field with (copy) added to it.

Alternatively, if you have used a derived field as a component for dimensions or metrics in your data view:

  1. Select the component. Note that the component might have a different name than your derived field.

  2. In the Component panel, select the Edit icon next to your derived field, underneath Schema field name.

  3. To edit your derived field, use the Edit derived field interface. See Derived field interface.

    • Select Save to save your updated derived field.

    • Select Cancel to cancel any changes you made to the derived field.

    • Select Save As to save the derived field as a new derived field. The new derived field has the same name as the original edited derived field with (copy) added to it.

Delete a derived field

  1. Select an existing Data view. See Data views for more information.

  2. Select the Components tab of the Data view.

  3. Select Schema fields tab in Connection pane.

  4. Select Derived fields > container.

  5. Hover over the derived field that you want to delete, and select Edit icon.

  6. In the Edit derived field interface, select Delete.

    A Delete component dialog asks you to confirm the deletion. Consider any external references there might exist to the derived field outside of the Data view.

    • Select Continue to delete the derived field.

Alternatively, if you have used a derived field as a component for dimensions or metrics in your data view:

  1. Select the component. Note that the component might have a different name than your derived field.

  2. In the Component panel, select the Edit icon next to your derived field, underneath Schema field name.

  3. In the Edit derived field interface, select Delete.

    A Delete component dialog asks you to confirm the deletion. Consider any external references there might exist to the derived field outside of the Data view.

    • Select Continue to delete the derived field.
NOTE

Derived fields are managed at a Connection level in Customer Journey Analytics. Any change made to a derived field in any of the Data views associated with that Connection applies across all these associated Data views.

Function templates

To quickly create a derived field for specific use cases, function templates are available. These function templates can be accessed from the selector area in the derived field interface or are presented upon first use in the Start with a field template wizard.

Marketing channels

This template is configured to use the Url Parse and Case When functions multiple times to get appropriate values from a URL. Logic is then applied on these values to associate the URL to a specific marketing channel.

 Details

To use the template, you have to specify the correct parameters for each function listed as part of the rules in the template. See Function reference for more information.

Screenshot of the Marketing channel template rule builder

Function reference

NOTE

You must have the Select package in order to use the functionality described in this section. Contact your administrator if you’re unsure which Customer Journey Analytics package you have.

For each supported function, find details below on:

  • specifications:

    • input data type: type of data supported,
    • input: possible values for input,
    • included operators: operators supported for this function (if any),
    • limitations: limitations that apply for this specific function,
    • output.
  • use cases, including:

    • data before defining the derived field,
    • how to define the derived field,
    • data after defining the derived field.
  • constraints (if applicable).

NOTE

The Lookup function has been renamed to Classify. See the Classify function for more information.

Case When

Applies conditionals, based on defined criteria from one or more fields. These criteria are then used to define the values in a new derived field, based on the sequence of the conditions.

 Details

Specifications

Input Data Type Input Included Operators Limitations Output
  • String
  • Numeric
  • Date
  • If, Else If container:

    • Value
      • Rules
      • Standard fields
      • Fields
    • Criterion (see included operators, based on selected value type)
  • Then set value to, Otherwise set value to:

    • Value
      • Rules
      • Standard fields
      • Fields

Strings

  • Equals
  • Equals any term
  • Contains the phrase
  • Contains any term
  • Contains all terms
  • Starts with
  • Starts with any term
  • Ends with
  • Ends with any term
  • Does not equal
  • Does not equal any term
  • Does not contain the phrase
  • Does not contain any term
  • Does not contain all terms
  • Does not start with
  • Does not start with any term
  • Does not end with
  • Does not end with any term
  • Is set
  • Is not set

Numeric

  • Equals
  • Does not equal
  • Is greater than
  • Is greater than or equal to
  • Is less than
  • Is less than or equal to
  • Is set
  • Is not set

Dates

  • Equals
  • Does not equal
  • Is later than
  • Is later than or equal to
  • Is before
  • Is before or equal to
  • Is set
  • Is not set
  • 5 functions per derived field
  • 200 operators per derived field. An example of one single operator is ‘Referring Domain contains google’.

New derived field

Use case 1

You want to define rules to identify various marketing channels, by applying cascading logic to set a marketing channel field to the proper value:

  • If the referrer is from a search engine and the page has a query string value where cid contains ps_, the marketing channel should be identified as a Paid Search.
  • If the referrer is from a search engine and the page does not have the query string cid, the marketing channel should be identified as a Natural Search.
  • If a page has a query string value where cid contains em_, the marketing channel should be identified as an Email.
  • If a page has a query string value where cid contains ds_, the marketing channel should be identified as a Display Ad.
  • If a page has a query string value where cid contains so_, the marketing channel should be identified as a Paid Social.
  • If the referrer is from a referring domain of twitter.com, facebook.com, linkedin.com, or tiktok.com, the marketing channel should be identified as a Natural Social.
  • If none of the above rules are matched, then the marketing channel should be identified as Other Referrer.

In case your site receives the following sample events, containing Referrer and Page URL, these events should be identified as follows:

Event Referrer Page URL Marketing Channel
1 https://facebook.com https://site.com/home Natural Social
2 https://abc.com https://site.com/?cid=ds_12345678 Display
3 https://site.com/?cid=em_12345678 Email
4 https://google.com https://site.com/?cid=ps_abc098765 Paid Search
5 https://google.com https://site.com/?cid=em_765544332 Email
6 https://google.com Natural Search

Data before

Referrer Page URL
https://facebook.com https://site.com/home
https://abc.com https://site.com/?cid=ds_12345678
https://site.com/?cid=em_12345678
https://google.com https://site.com/?cid=ps_abc098765
https://google.com https://site.com/?cid=em_765544332
https://google.com

Derived field

You define a new Marketing Channel derived field. You use the CASE WHEN functions to define rules that create values for the based on existing values for both the Page URL and Referring URL field.

Note the usage of the function URL PARSE to define rules to fetch the values for Page Url and Referring Url before the CASE WHEN rules are applied.

Screenshot of the Case when rule 1

Data after

Marketing Channel
Natural Social
Display
Email
Paid Search
Email
Natural Search

Use case 2

You have collected several different variations of search within your Product Finding Methods dimension. To understand the overall performance of search vs. browse, you must spend a great deal of time combining the results manually.

Your site collects the following values for your Product Finding Methods dimension. In the end, all of these values indicate a search.

Collected value Actual value
search p13n_no search
search p13n_yes search
search refine p13n_no search
search refine p13n_yes search
search redirect p13n_yes search
search-redirect search

Data before

Product Finding Methods
search p13_no
search p13_yes
browse
search refine p13_no
search refine p13_yes
browse
search redirect p13_yes
search-redirect
browse

Derived field

You define a Product Finding Methods (new) derived field. You create the following CASE WHEN rules in rule builder. These rules apply logic to all possible variations of the old Product Finding Methods field values for search and browse using the Contains the phrase criterion.

Screenshot of the Case When rule 2

Data after

Product Finding Methods (new)
search
search
browse
search
search
browse
search
search
browse

Use case 3

As a travel company, you would like to bucket trip duration for booked trips so you can report on bucketed lengths of trips.

Assumptions:

  • The organization is collecting trip duration into a numeric field.
  • They would like to bucket 1-3 day durations into a bucket called ‘short trip’
  • They would like to bucket 4-7 day durations into a bucket called ‘medium trip’
  • They would like to bucket 8+ day durations into a bucket called ‘long trip’
  • 132 trips were booked for a 1-day duration
  • 110 trips were booked for a 2-day duration
  • 105 trips were booked for a 3-day duration
  • 99 trips were booked for a 4-day duration
  • 92 trips were booked for a 5-day duration
  • 85 trips were booked for a 6-day duration
  • 82 trips were booked for a 7-day duration
  • 78 trips were booked for an 8-day duration
  • 50 trips were booked for a 9-day duration
  • 44 trips were booked for a 10-day duration
  • 38 trips were booked for an 11-day duration
  • 31 trips were booked for a 12-day duration

Your desired report should look like:

Trip Duration Type Bookings
medium trip 358
short trip 347
long trip 241

Data before

Trip Duration
1
12
3
6
4
8
6
2
1
2
21
8

Derived field

You define a Trip Duration (bucketed) derived field. You create the following CASE WHEN rule in rule builder. This rule applies logic to bucket the old Trip Duration field values into three values: short trip, medium trip, and long trip.

Screenshot of the Case When rule 3

Data after

Trip Duration (bucketed)
short trip
long trip
short trip
medium trip
medium trip
long trip
medium trip
short trip
short trip
short trip
long trip
long trip

More information

Customer Journey Analytics uses a nested container structure, modeled after Adobe Experience Platform’s XDM (Experience Data Model). See Containers and Filter containers for more background information. This container model, albeit flexible by nature, imposes some constraints when using the rule builder.

Customer Journey Analytics uses the following default container model:

The following constraints apply and are enforced when selecting and setting values.

Constraints
A Values you select within the same If, Else If construct (using And or Or) in a rule must originate from the same container and can be of any type (string String, numeric Numeric, and so forth).
Screenshot of dependency A
B All the values you set across a rule must be from the same container and have the same type or a derived value of the same type.
Screenshot of Dependency B
C The values you select across If, Else If constructs in the rule do not have to originate from the same container and do not have to be of the same type.
Screenshot of Dependency C

Classify

Defines a set of values that are replaced by corresponding values in a new derived field.

 Details
NOTE

This function was originally named Lookup but has been renamed to Classify to accommodate a forthcoming Lookup function with different functionality.

Specifications

Input Data Type Input Included Operators Limitations Output
  • String
  • Numeric
  • Date
  • Field to classify:
    • Rules
    • Standard fields
    • Fields
  • When value equals and Replace values with:

    • String
  • Show original values
    • Boolean

N/A

5 functions per derived field
100 rows per function

New derived field

Use case 1

You do have a CSV-file that includes a key column for hotelID and one or more additional columns associated with the hotelID: city, rooms, hotel name.
You are collecting Hotel ID in a dimension but would like to create a Hotel Name dimension derived from the hotelID in the CSV file.

CSV-file structure and content

hotelID city rooms hotel name
SLC123 Salt Lake City 40 SLC Downtown
LAX342 Los Angeles 60 LA Airport
SFO456 San Francisco 75 Market Street
AMS789 Amsterdam 50 Okura

Current Report

Hotel ID Product Views
SLC123 200
LX342 198
SFO456 190
AMS789 150

Desired Report

Hotel Name Product Views
SLC Downtown 200
LA Airport 198
Market Street 190

Data before

Hotel ID
SLC123
LAX342
SFO456
AMS789

Derived field

You define a Hotel Name derived field. You use the CLASSIFY function to define a rule where you can classify values of the Hotel ID field and replace with new values.

If you want to include original values that you have not defined as part of the values to classify (for example Hotel ID AMS789), ensure you select Show original values. This ensures AMS789 is part of the output for the derived field, despite that value not being classified.

Screenshot of the Classify rule 1

Data after

Hotel Name
SLC Downtown
LA Airport
Market Street

Use case 2

You have collected URLs instead of the friendly page name for several pages. This mixed collection of values breaks the reporting.

Data before

Page Name
Home Page
Flight Search
http://www.adobetravel.ca/Hotel-Search
https://www.adobetravel.com/Package-Search
Deals & Offers
http://www.adobetravel.ca/user/reviews
https://www.adobetravel.com.br/Generate-Quote/preview

Derived field

You define a Page Name (updated) derived field. You use the CLASSIFY function to define a rule where you can classify values of your existing Page Name field and replace with updated correct values.

Screenshot of the Classify rule 2

Data after

Page Name (updated)
Home Page
Flight Search
Hotel Search
Package Search
Deals & Offers
Reviews
Generate Quote

More information

The following additional functionality is available in the Classify rule interface:

  • To quickly clear all table values, select Erase Clear all table values.
  • To upload a CSV file containing original values for When values equal and new values for Replace values with, select CSV Upload CSV.
  • To download a template for creating a CSV file with original and new values to upload, select Download Download CSV template.
  • To download a CSV file with all original and new values populated in the rule interface, select Download Download CSV values.

Concatenate

Combines field values into a single new derived field with defined delimiters.

 Details

Specifications

Input Data Type Input Included Operators Limitations Output
  • String
  • Value:
    • Rules
    • Standard fields
    • Fields
    • String
  • Delimiter:
    • String

N/A

2 functions per derived field

New derived field

Use case

You currently collect origin and destination airport codes as separate fields. You would like to take the two fields and combine them into a single dimension separated by a hyphen (-). So you can analyze the combination of origin and destination to identify top routes booked.

Assumptions:

  • Origin and destination values are collected in separate fields in the same table.
  • The user determines to use the delimiter ‘-’ between the values.

Imagine the following bookings occur:

  • Customer ABC123 books a flight between Salt Lake City (SLC) and Orlando (MCO)
  • Customer ABC456 books a flight between Salt Lake City (SLC) and Los Angeles (LAX)
  • Customer ABC789 books a flight between Salt Lake City (SLC) and Seattle (SEA)
  • Customer ABC987 books a flight between Salt Lake City (SLC) and San Jose (SJO)
  • Customer ABC654 books a flight between Salt Lake City (SLC) and Orlando (MCO)

The desired report should look like:

Origin / Destination Bookings
SLC-MCO 2
SLC-LAX 1
SLC-SEA 1
SLC-SJO 1

Data before

Origin Destination
SLC MCO
SLC LAX
SLC SEA
SLC SJO
SLC MCO

Derived field

You define a new Origin - Destination derived field. You use the CONCATENATE function to define a rule to concatenate the Original and Destination fields using the - Delimiter.

Screenshot of the Concatenate rule

Data after

Origin - Destination
(derived field)
SLC-MCO
SLC-LAX
SLC-SEA
SLC-SJO
SLC-MCO

Find and Replace

Finds all values in a selected field and replaces those values with a different value in a new derived field.

 Details

Specifications

Input Data Type Input Included Operators Limitations Output
  • String
  • Value
    • Rules
    • Standard fields
    • Fields
  • Find all, and replace all with:
    • String

Strings

  • Find all, and replace all with

5 functions per derived field

New derived field

Use case

You have received some malformed values for your external marketing channels report, for example email%20 marketing instead of email marketing. These malformed values fracture your reporting and make it more difficult to see how email is performing. You want to replace email%20marketing with email marketing.

Original Report

External Marketing Channels Sessions
email marketing 500
email %20marketing 24

Preferred Report

External Marketing Channels Sessions
email marketing 524

Data before

External Marketing
email marketing
email%20marketing
email marketing
email marketing
email%20marketing

Derived field

You define an Email Marketing (updated) derived field. You use the FIND AND REPLACE function to define a rule to find and replace all occurrences of email%20marketing with email marketing.

Screenshot of the Find and Replace rule

Data after

External Marketing (updated)
email marketing
email marketing
email marketing
email marketing
email marketing

Lookup

Lookup values using a field from a lookup dataset and returns a value in a new derived field or for further rule processing.

 Details

Specification

Input Data Type Input Included Operators Limit Output
  • String
  • Numeric
  • Date
  • Field to apply lookup:
    • Rules
    • Standard fields
    • Fields
  • Lookup dataset
    • Dataset
  • Matching key
    • Rules
    • Fields
  • Values to return
    • Rules
    • Fields

N/A

3 functions per derived field

New derived field or value for further processing in next rule

Use case

You would like to lookup the activity name using the activity id collected when your customers clicked on a personalized banner shown through Adobe Target. You want to use a lookup dataset with Analytics for Target (A4T) activities containing activity ids and activity names.

A4T lookup dataset

Activity Id Activity Name
415851 MVT Test Category Pages
415852 Luma - Campaign Max 2022
402922 Home Page Banners

Derived field

You define an Activity Name derived field. You use the LOOKUP function to define a rule to lookup the value from your collected data, specified in the Field to apply lookup field (for example ActivityIdentifier). You select the lookup dataset from the Lookup dataset list (for example New CJA4T Activities). Then you selecting the identifier field (for example ActivityIdentifier) from the Matching key list and the field to return from the Values to return list (for example ActivityName).

Screenshot of the Lowercase rule

More info

You can quickly insert a Lookup function in the rule builder, already containing one or more other functions.

  1. Select Schema fields from selector.
  2. Select Schema field icon Lookup datasets.
  3. Select your lookup dataset and find the field you want to use for lookup.
  4. Drag and drop the lookup field on any of the available input fields for a function (for example Case When). When valid, a blue box, labeled + Add, allows you to drop the field and automatically insert a Lookup function before the function you dropped the lookup field on. The inserted Lookup function is automatically populated with relevant values for all fields.
    Lookup drag

Lowercase

Converts values from a field to lowercase and stores it into a new derived field.

 Details

Specification

Input Data Type Input Included Operators Limit Output
  • String
  • Numeric
  • Date
  • Field:
    • Rules
    • Standard fields
    • Fields

N/A

2 functions per derived field

New derived field

Use case

You would like to convert all collected product names into lowercase for proper reporting.

Data before

Collected Product Names Product Views
Tennis racket 35
Tennis Racket 33
tennis racket 21
Baseball bat 15
Baseball Bat 12
baseball bat 10

Derived field

You define a Product Names derived field. You use the LOWERCASE function to define a rule to convert the value from the Collected Product Names field to lowercase and store that in the new derived field.

Screenshot of the Lowercase rule

Data after

Product Names Product Views
tennis racket 89
baseball bat 37

Merge Fields

Merges values from two different fields into a new derived field.

 Details

Specification

Input Data Type Input Included Operators Limit Output
  • String
  • Numeric
  • Date
  • Field:
    • Rules
    • Standard fields
    • Fields

N/A

5 functions per derived field

New derived field

Use case

You would like to create a dimension made up from the page name field and the call reason field with the intent of analyzing the journey across channels.

Data before

Page Name Session Visitors
help page 250 200
home page 500 250
product detail page 300 200
Call Reason Session Visitors
questions about my order 275 250
make a change to my order 150 145
problem with ordering 100 95

Derived field

You define a Cross Channel Interactions derived field. You use the MERGE FIELDS function to define a rule to merge the values from the Page Name field and Call Reason field and store that in the new derived field.

Screenshot of the Merge Fields rule

Data after

Cross Channel Interactions Sessions Visitors
home page 500 250
product detail page 300 200
questions about my order 275 250
help page 250 200
make a change to my order 150 145
problem with ordering 100 95

More information

You must select the same type of fields within a Merge Fields rule. For example, if you select a Date field, all other fields you want to merge have to be Date fields.

Screenshot of constraint on merge fields

Regex Replace

Replaces a value from a field using a regular expression into a new derived field.

 Details

Specification

Input Data Type Input Included Operators Limit Output
  • String
  • Numeric
  • Field:
    • Rules
    • Standard fields
    • Fields
  • Regex:
    • String
  • Output Format:
    • String
  • Case sensitive
    • Boolean

N/A

1 function per derived field

New derived field

Use case

You would like to grab a potion of a URL and use that as a unique page identifier to analyze traffic. You use [^/]+(?=/$|$) for the regular expression to capture the end of the URL and $1 as the output pattern.

Data before

Page URL
https://business.adobe.com/products/analytics/adobe-analytics-benefits.html
https://business.adobe.com/products/analytics/adobe-analytics.html
https://business.adobe.com/products/experience-platform/customer-journey-analytics.html
https://business.adobe.com/products/experience-platform/adobe-experience-platform.html

Derived field

You create a Page Identifier derived field. You use the REGEX REPLACE function to define a rule to replace value of the Referring URL field using a Regex of [^/]+(?=/$|$) and Output format of $1.

Screenshot of the Regex Replac rule

Data after

Page Identifier
adobe-analytics-benefits.html
adobe-analytics.html
customer-journey-analytics.html
adobe-experience-platform.html

More information

Customer Journey Analytics uses a subset of the Perl regex syntax. The following expressions are supported:

Expression Description
a A single character a.
a|b A single character a or b.
[abc] A single character a, b, or c.
[^abc] Any single character except a, b, or c.
[a-z] Any single character in the range of a-z.
[a-zA-Z0-9] Any single character in the range of a-z, A-Z, or digits 0-9.
^ Matches the beginning of the line.
$ Matches the end of the line.
\A Start of string.
\z End of string.
. Matches any character.
\s Any whitespace character.
\S Any non-whitespace character.
\d Any digit.
\D Any non-digit.
\w Any letter, number, or underscore.
\W Any non-word character.
\b Any word boundary.
\B Any character that is not a word boundary.
\< Start of word.
\> End of word.
(...) Capture everything enclosed.
(?:...) Non-marking capture. Prevents the match from being referenced in the output string.
a? Zero or one of a.
a* Zero or more of a.
a+ One ore more of a.
a{3} Exactly 3 of a.
a{3,} 3 or more of a.
a{3,6} Between 3 and 6 of a.

You can use these sequences in the Output format any number of times and in any order to achieve the desired string output.

Output placeholder sequence Description
$& Outputs what matched the whole expression.
$n Outputs what matched the nth sub-expression. For example, $1 outputs the first sub-expression.
$` Outputs the text between the end of the last match found (or the start of the text if no previous match was found), and the start of the current match.
$+ Outputs what matched the last marked sub-expression in the regular expression.
$$ Outputs the string character "$".

Split

Splits a value from a field into a new derived field.

 Details

Specification

Input Data Type Input Included Operators Limit Output
  • String
  • Numeric
  • Field:
    • Rules
    • Standard fields
    • Fields
  • Method:
    • From the left
    • From the right
    • Convert to array
  • For Delimiter:
    • String
  • For Index:
    • Numeric

N/A

5 functions per derived field

New derived field

Use case 1

You collect voice app responses into a delimited list in a single dimension. You would like each value in the list to be a unique value in the responses report.

Data before

Voice App Responses Events
it was great,made perfect sense,will recommend to others 1
it was great,somewhat confusing,will recommend to others 1
it was not great,very confusing,will not recommned to others 1

Derived field

You create a Responses derived field. You use the SPLIT function to define a rule to use the Convert to array method to convert the values from the Voice App Response field using , as the Delimiter.

Screenshot of the Split rule 1

Data after

Responses Events
it was great 2
will recommend to others 2
it was not great 1
made perfect sense 1
somewhat confusing 1
very confusing 1
will not recommend to others 1

Use case 2

You collect voice app responses into a delimited list in a single dimension. You would like the responses from the first value in the list into its own dimension. You would like to put the last value in the list into its own dimension.

Data before

Responses Events
it was great,made perfect sense,will recommed to others 1
it was great,somewhat confusing,will recommend to others 1
it was not great,very confusing,will not recommned to others 1

Derived field

You create a First Response derived field. You use the SPLIT function to define a rule to take the first value from the Responses field from the left of the response , as the delimiter.

Screenshot of the Split rule - first value

You create a Second Response derived field to take the last value from the Responses field by selecting From the right, 1 as the Delimiter and 1 as the Index.

Screenshot of the Split rule - last value

Data after

First Response Events
it was great 2
it was not great 1
Second Response Events
will recommend to others 2
will not recommend to others 1

Trim

Trims whitespace, special characters, or number of characters from either the beginning or the end of field values into a new derived field.

 Details

Specification

Input Data Type Input Included Operators Limit Output
  • String
  • Field
    • Rules
    • Standard fields
    • Fields
  • Trim whitespace
  • Trim special characters
    • Input of special characters
  • Trim from left
    • From 
      • String start
      • Position
        • Position #
      • String
        • String value
        • Index
        • Flag to include string
    • To
      • String end
      • Position
        • Position #
      • String
        • String value
        • Index
        • Flag to include string
      • Length
  • Trim from right
    • From 
      • String end
      • Position
        • Position #
      • String
        • String value
        • Index
        • Flag to include string
    • To
      • String start
      • Position
        • Position #
      • String
        • String value
        • Index
        • Flag to include string
      • Length

N/A

1 function per derived field

New derived field

Use case 1

You collect product data, however that data contains hidden whitespace characters which fragment reporting. You would like to easily trim any excess whitespace

Data before

Product ID Events
"prod12356 " 1
"prod12356" 1
" prod12356" 1

Derived field

You create a Product Identifier derived field. You use the TRIM function to define a rule to Trim whitespace from the Product ID field.

Screenshot of the Split rule 1

Data after

Product Identifier Events
"prod12356" 3

Use case 2

Your data on page names collected includes some erroneous special characters at the end of the page name which must be removed.

Data before

Name Events
home page# 1
home page? 1
home page% 1
home page& 1
home page/ 1

Derived field

You create a Page Name derived field. You use the TRIM function to define a rule to Trim special characters from the Name field using the Special characters #?%&/.

Screenshot of the Split rule - first value

Data after

Page Name Events
home page 5

Use case 3

You collect data including a storeID. The storeID contains the abbreviated US state code as the first two characters. You want to only use that state code in your reporting.

Data before

storeID Events
CA293842 1
CA423402 1
UT123418 1
UT189021 1
ID028930 1
OR234223 1
NV22342 1

Derived field

You create a Store Identifier derived field. You use the TRIM function to define a rule to Truncate from right the storeID field from String end to position 3.

Screenshot of the Split rule - first value

Data after

Store Identifier Events
CA 2
UT 2
ID 1
OR 1
NV 1

URL Parse

Parses out different parts of a URL including protocol, host, path, or query parameters.

 Details

Specifications

Input Data Type Input Included Operators Limit Output
  • String
  • Field:
    • Rules
    • Standard fields
    • Fields
  • Option:
    • Get protocol
    • Get host
    • Get path
    • Get query string value
      • Query parameter:
        • String
    • Get hash value

N/A

5 functions per derived field

New derived field

Use case 1

You only want use the referring domain from the referring URL as part of a marketing channel’s set of rules.

Data before

Referring URL
https://www.google.com/
https://duckduckgo.com/
https://t.co/
https://l.facebook.com/

Derived field

You define a Referring Domain derived field. You use the URL PARSE function to define a rule to fetch the host from the Referring URL field and store that in the new derived field.

Screenshot of the Url Parse rule 1

Data after

Referrer Domain
www.google.com
duckduckgo.com
t.co
l.facebook.com

Use case 2

You want to use the value of the cid parameter of a query string in a Page URL as part of the output of a derived tracking code report.

Data before

Page URL
https://www.adobe.com/?cid=abc123
https://www.adobe.com/?em=email1234&cid=def123
https://www.adobe.com/landingpage?querystring1=test&test2=1234&cid=xyz123

Derived field

You define a Query String CID derived field. You use the URL PARSE function to define a rule to fetch the value of the query string parameter in the Page URL field, specifying cid as the query parameter. The output value is stored in the new derived field.

Screenshot of the Url Parse rule 2

Data after

Query String CID
abc123
def123
xyz123

Limitations

The following limitations apply to the Derived field functionality in general:

  • You can use a maximum of ten different schema fields (not including standard fields) when defining rules for a derived field.
    • From this maximum of ten different schema fields, only a maximum of three lookup schema or profile schema fields are allowed.
  • You can have a maximum of 100 derived fields per Customer Journey Analytics connection.

Summary of function limitations

Function Limitations

Case When

  • 5 Case When functions per derived field
  • 200 operators per derived field

Classify

  • 5 Classify functions per derived field
  • 100 rows per function

Concatenate

  • 2 Concatenate functions per derived field

Find & Replace

  • 2 Find & Replace functions per derived field

Lookup

  • 5 Lookup functions per derived field

Lowercase

  • 2 Lowercase functions per derived field

Merge Fields

  • 2 Merge Fields functions per derived field

Regex Replace

  • 1 Regex Replace function per derived field

Split

  • 5 Split functions per derived field

Trim

  • 1 Trim function per derived field

URL Parse

  • 5 URL Parse functions per derived field

Operators

An operator in an If or Else If construct within a Case When function is the combination of a criterion with one value. Every additional value for the criterion adds to the number of operators.

As an example, the condition below uses 13 operators.

Sample operators

More information

Trim and Lowercase are features already available in the component settings in Data views. Using Derived Fields allows you to combine these functions to do more complex data transformation directly in Customer Journey Analytics. For example, you can use Lowercase to remove case sensitivity in an event field, and then use Lookup to match the new lowercase field to a lookup dataset that only has lookup keys in lowercase. Or you can use Trim to remove characters before setting up Lookup on the new field.

Support for lookup and profile fields in Derived Fields enables you to transform data based on event lookups and profile attributes. This can be especially helpful in B2B scenarios with account-level data in lookup or profile datasets. Additionally, this support is useful to manipulate data in common fields from lookup data (like campaign info and offer type), or from profile data (like member tier and account type).

See for more background information on Derived Fields:

On this page