Metrics endpoint

Observability metrics provide insights into usage statistics, historical trends, and performance indicators for various features in Adobe Experience Platform. The /metrics endpoint in the Observability Insights API allows you to programmatically retrieve metric data for your organization’s activity in Platform.

NOTE

The previous version of the metrics endpoint (V1) has been deprecated. This document focuses exclusively on the current version (V2). For details on the V1 endpoint for legacy implementations, please refer to the API reference.

Getting started

The API endpoint used in this guide is part of the Observability Insights API. Before continuing, please review the getting started guide for links to related documentation, a guide to reading the sample API calls in this document, and important information regarding required headers that are needed to successfully make calls to any Experience Platform API.

Retrieve observability metrics

You can retrieve metrics data by making a POST request to the /metrics endpoint, specifying the metrics you wish to retrieve in the payload.

API format

POST /metrics

Request

curl -X POST \
  https://platform.adobe.io/data/infrastructure/observability/insights/metrics \
  -H 'Authorization: Bearer {ACCESS_TOKEN}' \
  -H 'x-api-key: {API_KEY}' \
  -H 'x-gw-ims-org-id: {IMS_ORG}' \
  -H 'x-sandbox-name: {SANDBOX_NAME}' \
  -d '{
        "start": "2020-07-14T00:00:00.000Z",
        "end": "2020-07-22T00:00:00.000Z",
        "granularity": "day",
        "metrics": [
          {
            "name": "timeseries.ingestion.dataset.recordsuccess.count",
            "filters": [
              {
                "name": "dataSetId",
                "value": "5edcfb2fbb642119194c7d94|5eddb21420f516191b7a8dad",
                "groupBy": true
              }
            ],
            "aggregator": "sum",
            "downsample": "sum"
          },
          {
            "name": "timeseries.ingestion.dataset.dailysize",
            "filters": [
              {
                "name": "dataSetId",
                "value": "5eddb21420f516191b7a8dad",
                "groupBy": false
              }
            ],
            "aggregator": "sum",
            "downsample": "sum"
          }
        ]
      }'
Property Description
start The earliest date/time from which to retrieve metric data.
end The latest date/time from which to retrieve metric data.
granularity An optional field that indicates the time interval to divide metric data by. For example, a value of DAY returns metrics for each day between the start and end date, whereas a value of MONTH would group metric results by month instead. When using this field, a corresponding downsample property must also be provided to indicate the aggregation function by which to group data.
metrics An array of objects, one for each metric you want to retrieve.
name The name of a metric recognized by Observability Insights. See the appendix for a full list of accepted metric names.
filters An optional field that allows you to filter metrics by specific datasets. The field is an array of objects (one for each filter), with each object containing the following properties:
  • name: The type of entity to filter metrics against. Currently, only dataSets is supported.
  • value: The ID of one or more datasets. Multiple dataset IDs can be provided as a single string, with each ID separated by vertical bar characters (|).
  • groupBy: When set to true, indicates that the corresponding value represents multiple datasets whose metric results should be returned separately. If set to false, metric results for those datasets are grouped together.
aggregator Specifies the aggregation function that should be used to group multiple times-series records into single results. For detailed information on available aggregators, refer to the OpenTSDB documentation.
downsample An optional field that allows you to specify an aggregation function to reduce the sampling rate of metric data by sorting fields into intervals (or “buckets”). The interval for the downsampling is determined by the granularity property. For detailed information on downsampling, refer to the OpenTSDB documentation.

Response

A successful response returns the resulting datapoints for the metrics and filters specified in the request.

{
  "metricResponses": [
    {
      "metric": "timeseries.ingestion.dataset.recordsuccess.count",
      "filters": [
        {
          "name": "dataSetId",
          "value": "5edcfb2fbb642119194c7d94|5eddb21420f516191b7a8dad",
          "groupBy": true
        }
      ],
      "datapoints": [
        {
          "groupBy": {
            "dataSetId": "5edcfb2fbb642119194c7d94"
          },
          "dps": {
            "2020-07-14T00:00:00Z": 44.0,
            "2020-07-15T00:00:00Z": 46.0,
            "2020-07-16T00:00:00Z": 36.0,
            "2020-07-17T00:00:00Z": 50.0,
            "2020-07-18T00:00:00Z": 38.0,
            "2020-07-19T00:00:00Z": 40.0,
            "2020-07-20T00:00:00Z": 42.0,
            "2020-07-21T00:00:00Z": 42.0,
            "2020-07-22T00:00:00Z": 50.0
          }
        },
        {
          "groupBy": {
            "dataSetId": "5eddb21420f516191b7a8dad"
          },
          "dps": {
            "2020-07-14T00:00:00Z": 44.0,
            "2020-07-15T00:00:00Z": 46.0,
            "2020-07-16T00:00:00Z": 36.0,
            "2020-07-17T00:00:00Z": 50.0,
            "2020-07-18T00:00:00Z": 38.0,
            "2020-07-19T00:00:00Z": 40.0,
            "2020-07-20T00:00:00Z": 42.0,
            "2020-07-21T00:00:00Z": 42.0,
            "2020-07-22T00:00:00Z": 50.0
          }
        }
      ],
      "granularity": "DAY"
    },
    {
      "metric": "timeseries.ingestion.dataset.dailysize",
      "filters": [
        {
          "name": "dataSetId",
          "value": "5eddb21420f516191b7a8dad",
          "groupBy": false
        }
      ],
      "datapoints": [
        {
          "groupBy": {},
          "dps": {
            "2020-07-14T00:00:00Z": 38455.0,
            "2020-07-15T00:00:00Z": 40213.0,
            "2020-07-16T00:00:00Z": 31476.0,
            "2020-07-17T00:00:00Z": 43705.0,
            "2020-07-18T00:00:00Z": 33227.0,
            "2020-07-19T00:00:00Z": 34977.0,
            "2020-07-20T00:00:00Z": 36735.0,
            "2020-07-21T00:00:00Z": 36737.0,
            "2020-07-22T00:00:00Z": 43715.0
          }
        }
      ],
      "granularity": "DAY"
    }
  ]
}
Property Description
metricResponses An array whose objects represent each of the metrics specified in the request. Each object contains information about the filter configuration and returned metric data.
metric The name of one of the metrics provided in the request.
filters The filter configuration for the specified metric.
datapoints An array whose objects represent the results of the specified metric and filters. The number of objects in the array depends on the filter options provided in the request. If no filters were provided, the array will only contain a single object that represents all datasets.
groupBy If multiple datasets were specified in the filter property for a metric, and the groupBy option was set to true in the request, this object will contain the ID of the dataset that the corresponding dps property applies to.

If this object appears empty in the response, the corresponding dps property applies to all datasets provided in the filters array (or all datasets in Platform if no filters were provided).
dps The returned data for the given metric, filter, and time range. Each key in this object represents a timestamp with a corresponding value for the specified metric. The time period between each datapoint depends on the granularity value specified in the request.

Appendix

The following section contains additional information about working with the /metrics endpoint.

Available metrics

The following tables list all of the metrics that are exposed by Observability Insights, broken down by Platform service. Each metric includes a description and accepted ID query parameter.

NOTE

All listed ID query parameters are optional unless stated otherwise.

Data Ingestion

The following table outlines metrics for Adobe Experience Platform Data Ingestion. Metrics in bold are streaming ingestion metrics.

Insights metric Description ID query parameter
timeseries.ingestion.dataset.new.count Total number of datasets created. N/A
timeseries.ingestion.dataset.size Cumulative size of all data ingested for one dataset for or all datasets. Dataset ID
timeseries.ingestion.dataset.dailysize Size of data ingested on a daily usage basis for one dataset or for all datasets. Dataset ID
timeseries.ingestion.dataset.batchfailed.count Number of batches failed for one dataset or for all datasets. Dataset ID
timeseries.ingestion.dataset.batchsuccess.count Number of batches ingested for one dataset or for all datasets. Dataset ID
timeseries.ingestion.dataset.recordsuccess.count Number of records ingested for one dataset or for all datasets. Dataset ID
timeseries.data.collection.validation.total.messages.rate Total number of messages for one dataset or for all datasets. Dataset ID
timeseries.data.collection.validation.valid.messages.rate Total number of valid messages for one dataset or for all datasets. Dataset ID
timeseries.data.collection.validation.invalid.messages.rate Total number of invalid messages for one dataset or for all datasets. Dataset ID
timeseries.data.collection.validation.category.type.count Total number of invalid “type” messages for one dataset or for all datasets. Dataset ID
timeseries.data.collection.validation.category.range.count Total number of invalid “range” messages for one dataset or for all datasets. Dataset ID
timeseries.data.collection.validation.category.format.count Total number of invalid “format” messages for one dataset or for all datasets. Dataset ID
timeseries.data.collection.validation.category.pattern.count Total number of invalid “pattern” messages for one dataset or for all datasets. Dataset ID
timeseries.data.collection.validation.category.presence.count Total number of invalid “presence” messages for one dataset or for all datasets. Dataset ID
timeseries.data.collection.validation.category.enum.count Total number of invalid “enum” messages for one dataset or for all datasets. Dataset ID
timeseries.data.collection.validation.category.unclassified.count Total number of invalid “unclassified” messages for one dataset or for all datasets. Dataset ID
timeseries.data.collection.validation.category.unknown.count Total number of invalid “unknown” messages for one dataset or for all datasets. Dataset ID
timeseries.data.collection.inlet.total.messages.received Total number of messages received for one data inlet or for all data inlets. Inlet ID
timeseries.data.collection.inlet.total.messages.size.received Total size of data received for one data inlet or for all data inlets. Inlet ID
timeseries.data.collection.inlet.success Total number of successful HTTP calls to one data inlet or to all data inlets. Inlet ID
timeseries.data.collection.inlet.failure Total number of failed HTTP calls to one data inlet or to all data inlets. Inlet ID

Identity Service

The following table outlines metrics for Adobe Experience Platform Identity Service.

Insights metric Description ID query parameter
timeseries.identity.dataset.recordsuccess.count Number of records written to their data source by Identity Service, for one dataset or all datasets. Dataset ID
timeseries.identity.dataset.recordfailed.count Number of records failed by Identity Service, for one dataset or for all datasets. Dataset ID
timeseries.identity.dataset.namespacecode.recordsuccess.count Number of Identity records successfully ingested for a namespace. Namespace ID (Required)
timeseries.identity.dataset.namespacecode.recordfailed.count Number of Identity records failed by a namespace. Namespace ID (Required)
timeseries.identity.dataset.namespacecode.recordskipped.count Number of Identity records skipped by a namespace. Namespace ID (Required)
timeseries.identity.graph.imsorg.uniqueidentities.count Number of unique identities stored in the identity graph for your IMS Organization. N/A
timeseries.identity.graph.imsorg.namespacecode.uniqueidentities.count Number of unique identities stored in the identity graph for a namespace. Namespace ID (Required)
timeseries.identity.graph.imsorg.numidgraphs.count Number of unique graph identities stored in the identity graph for your IMS Organization. N/A
timeseries.identity.graph.imsorg.graphstrength.uniqueidentities.count Number of unique identities stored in the identity graph for your IMS Organization for a particular graph strength (“unknown”, “weak”, or “strong”). Graph strength (Required)

Privacy Service

The following table outlines metrics for Adobe Experience Platform Privacy Service.

Insights metric Description ID query parameter
timeseries.gdpr.jobs.totaljobs.count Total number of jobs created from GDPR. ENV (Required)
timeseries.gdpr.jobs.completedjobs.count Total number of completed jobs from GDPR. ENV (Required)
timeseries.gdpr.jobs.errorjobs.count Total number of error jobs from GDPR. ENV (Required)

Query Service

The following table outlines metrics for Adobe Experience Platform Query Service.

Insights metric Description ID query parameter
timeseries.queryservice.query.scheduleonce.count Total number of non-recurring scheduled queries. N/A
timeseries.queryservice.query.scheduledrecurring.count Total number of recurring scheduled queries. N/A
timeseries.queryservice.query.batchquery.count Total number of executed batch queries. N/A
timeseries.queryservice.query.scheduledquery.count Total number of executed scheduled queries. N/A
timeseries.queryservice.query.interactivequery.count Total number of executed interactive queries. N/A
timeseries.queryservice.query.batchfrompsqlquery.count Total number of executed batch queries from PSQL. N/A

Real-time Customer Profile

The following table outlines metrics for Real-time Customer Profile.

Insights metric Description ID query parameter
timeseries.profiles.dataset.recordread.count Number of records read from the Data Lake by Profile, for one dataset or for all datasets. Dataset ID
timeseries.profiles.dataset.recordsuccess.count Number of records written to their data source by Profile, for one dataset or for all datasets. Dataset ID
timeseries.profiles.dataset.recordfailed.count Number of records failed by Profile, for one dataset or for all datasets. Dataset ID
timeseries.profiles.dataset.batchsuccess.count Number of Profile batches ingested for a dataset or for all datasets. Dataset ID
timeseries.profiles.dataset.batchfailed.count Number of Profile batches failed for one dataset or for all datasets. Dataset ID
platform.ups.ingest.streaming.request.m1_rate Incoming Request rate. IMS Org (Required)
platform.ups.ingest.streaming.access.put.success.m1_rate Ingestion success rate. IMS Org (Required)
platform.ups.ingest.streaming.records.created.m15_rate Rate of new records ingested for a dataset. Dataset ID (Required)
platform.ups.ingest.streaming.request.error.created.outOfOrder.m1_rate Rate of out-of-order timestamped records for create request for a dataset. Dataset ID (Required)
platform.ups.profile-commons.ingest.streaming.dataSet.record.created.timestamp Timestamp for last create record request for a dataset. Dataset ID (Required)
platform.ups.ingest.streaming.request.error.updated.outOfOrder.m1_rate Rate of out-of-order timestamped records for update request for a dataset. Dataset ID (Required)
platform.ups.profile-commons.ingest.streaming.dataSet.record.updated.timestamp Timestamp for last update record request for a dataset. Dataset ID (Required)
platform.ups.ingest.streaming.record.size.m1_rate Average record size. IMS Org (Required)
platform.ups.ingest.streaming.records.updated.m15_rate Rate of update requests for records ingested for a dataset. Dataset ID (Required)

Error messages

Responses from the /metrics endpoint may return error messages under certain conditions. These error messages are returned in the following format:

{
    "type": "http://ns.adobe.com/aep/errors/INSGHT-1000-400",
    "title": "Bad Request - Start date cannot be after end date.",
    "status": 400,
    "report": {
        "tenantInfo": {
            "sandboxName": "prod",
            "sandboxId": "49f58060-5d47-34rd-aawf-a5384333ff12",
            "imsOrgId": "{IMS_ORG}"
        },
        "additionalContext": null
    },
    "error-chain": [
        {
            "serviceId": "INSGHT",
            "errorCode": "INSGHT-1000-400",
            "invokingServiceId": "INSGHT",
            "unixTimeStampMs": 1602095177129
        }
    ]
}
Property Description
title A string containing the error message and the potential reason why it may have occurred.
report Contains contextual information about the error, including the sandbox and IMS Org being used in the operation that triggered it.

The following table lists the different error codes that can be returned by the API:

Error code Title Description
INSGHT-1000-400 Bad request payload Something was wrong with the request payload. Ensure that you match the payload formatting exactly as shown above. Any of the possible reasons can trigger this error:
  • Missing required fields such as aggregator
  • Invalid metrics
  • The request contains an invalid aggregator
  • A start date takes place after an end date
INSGHT-1001-400 Metrics query failed There was an error when attempting to query the metrics database, due to a bad request or the query itself being unparsable. Ensure that your request is properly formatted before trying again.
INSGHT-1001-500 Metrics query failed There was an error when attempting to query the metrics database, due to a server error. Try the request again, and if the problem persists, contact Adobe support.
INSGHT-1002-500 Service error The request could not be processed due to an internal error. Try the request again, and if the problem persists, contact Adobe support.
INSGHT-1003-401 Sandbox validation error The request could not be processed due to a sandbox validation error. Ensure that the sandbox name you provided in the x-sandbox-name header represents a valid, enabled sandbox for your IMS Organization before trying the request again.

On this page