DataStream 1 Configuration API v1

Create and manage DataStream 1 configurations to send logs about edge request-response cycles to selected destinations.

Learn more:


Overview

Akamai’s infrastructure is constantly gathering log entries from thousands of edge servers around the world. You can use the DataStream 1 Configuration API to capture these logs and deliver them to the destination of your choice at low latency.

The DataStream 1 Configuration API lets you create, update, delete, and view data streams for your properties to provide low latency streaming of data in raw and aggregated forms. You can configure a data stream to deliver log data to one or more destinations or integrate it with third-party web services or your own solution to access the data through the DataStream 1 Pull API.

Who should use this API

This API offers a programmatic alternative to the features of DataStream 1 available in Akamai Control Center.

Use this API service to monitor request-response cycles delivered through the Akamai platform. Either send the raw or aggregated log data to a destination or retrieve it with the DataStream 1 Pull API.

You can use log data for:

  • Monitoring. You can configure your data streams to send raw logs every 30 seconds or aggregated logs as often as every minute. This low latency streaming can help you proactively monitor performance, detect, and quickly resolve performance degradations.

  • Usage tracking. You can monitor for usage spikes to avoid exceeding commits.

  • Diagnostics. You can receive pre-aggregated metrics over a specific window of time. Use this option to easily switch between aggregated and raw data views for diagnostics or root cause analysis.

  • Troubleshooting. After you’ve detected an issue, you may need the necessary data to investigate and isolate the root cause. The log data you receive can help enable long-term trend analysis.

Get started

To configure this API for the first time:

  • Contact your Akamai representative to enable the DataStream module for your account.

  • Review Get started with APIs for details on how to set up client tokens to access any Akamai API. These tokens appear as custom hostnames that look like this: https://akzz-XXXXXXXXXXXXXXXX-XXXXXXXXXXXXXXXX.luna.akamaiapis.net.

  • To enable this API, choose the API service called Datastream Config and set the access level to READ-WRITE.

  • The API limits the scope of the returned data by the user’s account, its permissions, and access to business objects that are associated with the user and account.

API concepts

This section provides a road map of all the conceptual objects you come across when interacting with the DataStream 1 Configuration API and provides pointers to where you can learn more.

  • Groups. Each account features a hierarchy of groups, which control access to properties. Using either Control Center or the Identity Management: User Administration API, account administrators can assign properties to specific groups, each with its own set of users and accompanying roles. Your access to any given property depends on the role set for you in its group. Typically, you need a group identifier to create a data stream configuration. See the Group object.

  • Contracts. Each account features one or more contracts, each of which has a fixed term of service. You need a contract identifier to crate a data stream configuration. Typically, you need a contract identifier to create a data stream configuration.

  • Property. A property, also referred to as a configuration, provides the main way to control how edge servers respond to various kinds of requests for those assets. Properties apply rules to a set of hostnames, and you can only apply one property at a time to any given hostname. Data streams let you monitor the traffic served by a property. Also, you can monitor one property in up to 15 data streams. See the Property object.

  • Connector. A connector, also known as a destination, represents a third-party configuration where you can send the stream’s logs. You can configure your data stream to send logs to one or more destinations. DataStream 1 supports these connectors: Amazon S3, Datadog, Splunk, Sumo Logic, and custom HTTPS URL.

  • Stream. Collects logs about edge request-response cycles for one or more properties. Based on the type of selected data set parameters, it either streams raw data or aggregated metrics to destinations or makes them available through the DataStream 1 Pull API. It lets you control data set parameters you monitor in your logs and delivery conditions for these logs. You can update a data stream through versioning. In a data stream, you can monitor up to 15 properties. See the Stream configuration and Stream version objects.

  • Stream mode. Specifies whether your data stream sends or pulls log data.

    • By specifying destinations in the connectors array in a data stream configuration, you create a data stream that sends data to a specific destination for storing, monitoring, and analytical purposes. You can also access your log data with the DataStream 1 Pull API for up to 12 hours.

    • By sending an empty connectors array in a data stream configuration, you create a data stream that doesn’t send log data to any specific destination but makes it available to your integrations through the DataStream 1 Pull API for up to 12 hours.

  • Data sets. Groups parameters that you can either monitor in each request-response cycle or use as aggregation metrics for your logs. Selecting data set parameters lets you control the request-response cycle data that your data stream collects and sends to a destination or retains. See the Dataset object. These are available data set types:

    • RAW data sets provide information about individual edge request-response cycles on the Akamai platform. You can use raw data logs to find details about specific incidents, search the logs for instances using a specific IP address, or analyze the patterns of multiple attacks.

    • AGGREGATED data sets provide information about edge request-response cycles aggregated over selected metrics and a time window. You can use aggregated logs to search for the root cause of an error and monitor the performance, security status, and general behavior of your application.

  • DataStream 1 Pull API. A reporting API that lets you access log data collected by your data streams. Regardless of the stream mode you selected, this API stores log data for up to 12 hours.

Version management

Every time you edit a data stream, you create a version with an incremented version number. This lets you quickly adapt your existing data streams to collect logs for different properties, modify data set parameters they monitor, or change destinations where they send logs.

Each version that you activate becomes a default version for a data stream with the same activation status as its base version. For example, when you edit version 1 of an active data stream, you create an active version 2 that replaces the previous version on the production network. Conversely, by editing version 1 of an inactive data stream, you create an inactive version 2 of this data stream that replaces the previous version.

You can view and compare all configuration versions within a data stream, but you can only manage the activation status of the latest version. You also can’t revert a data stream to any previous version.

Workflow

Follow this basic workflow to configure a data stream:

  1. Determine the properties within the selected product that you want to monitor in the data stream. See List properties.

  2. Make sure your properties are active on the production network. See Activate a property in Property Manager. Note that you can only collect logs for active properties.

  3. Build one or more connector objects for the connectors array in the data stream configuration. These are the destinations where you want the data stream to deliver logs. See Connector in API concepts. You can also send an empty connectors array to retrieve the logs with the DataStream 1 Pull API. See Stream mode in API concepts.

  4. Get and store the data set parameters that you want the data stream to collect. See List data set parameters.

  5. Create and configure a data stream. See Create a stream.

  6. Enable the DataStream behavior in each property that’s part of the data stream. See Datastream in Property Manager. Note that you can only collect logs for DataStream-enabled properties that are active on the production network. You can also further control how your data stream collects logs in Property Manager. For each property, you can specify the streams that you want to receive logs from or configure the behavior in a custom rule to log requests matching your criteria. See Criteria in Property Manager.

Rate limiting

The DataStream 1 Configuration API limits each client to 20 requests per minute. Exceeding this limit results in a 429 error response. Consider this when calling successive operations as part of a loop.

All responses specify these rate limit headers:

  • X-RateLimit-Limit: The maximum number of tokens allowed.

  • X-RateLimit-Remaining: The number of tokens remaining. Except for
    any subsequent requests that reduce the number, this gradually
    increments until it reaches the X-RateLimit-Limit.

  • X-RateLimit-Next: If the X-RateLimit-Remaining has reached 0,
    this ISO 8601 timestamp indicates when you can next make an
    additional request.

Resources

This section provides details on the DataStream 1 Configuration API’s various operations and parameters.

API summary

Download the RAML descriptors for this API.

Operation Method Endpoint
List properties GET /datastream-config-api/v1/datastream1/properties/group/{groupId}
List data set parameters GET /datastream-config-api/v1/datastream1/datasets/{type}
List groups GET /datastream-config-api/v1/datastream1/groups
List streams GET /datastream-config-api/v1/datastream1/streams{?groupId}
Create a stream POST /datastream-config-api/v1/datastream1/streams
View a stream’s version GET /datastream-config-api/v1/datastream1/streams/{streamId}{?version}
Edit a stream PUT /datastream-config-api/v1/datastream1/streams/{streamId}
Delete a stream DELETE /datastream-config-api/v1/datastream1/streams/{streamId}
Activate a stream PUT /datastream-config-api/v1/datastream1/streams/{streamId}/activate
Deactivate a stream PUT /datastream-config-api/v1/datastream1/streams/{streamId}/deactivate

List properties

Returns properties that are available within the group.

GET /datastream-config-api/v1/datastream1/properties/group/{groupId}

Sample: /datastream-config-api/v1/datastream1/properties/group/12345

Parameter Type Sample Description
URL path parameters
groupId Integer 12345 Identifies the group.

Status 200 application/json

Object type: Property

Download schema: Properties.json

Response body:

[
    {
        "propertyId": 382631,
        "propertyName": "property1.com"
    },
    {
        "propertyId": 347459,
        "propertyName": "property2.com"
    },
    {
        "propertyId": 389150,
        "propertyName": "property3.com"
    }
]
  1. Run the List groups operation to find the relevant groupId.

  2. Make a GET request to /datastream-config-api/v1/datastream1/properties/group/{groupId}.

  3. From the result list, you can choose properties that you can enable log collection for in a data stream configuration. See the Create a stream or Edit a stream operations.

List data set parameters

Returns groups of data set parameters available. You can select the parameters that you want to monitor in the logs collected by your data stream configuration. See the Create a stream or Edit a stream operations.

GET /datastream-config-api/v1/datastream1/datasets/{type}

Sample: /datastream-config-api/v1/datastream1/datasets/RAW

Parameter Type Sample Description
URL path parameters
type String RAW The type of data set parameters, either RAW or AGGREGATED. See data sets in API concepts.

Status 200 application/json

Object type: Dataset

Download schema: Datasets.json

Response body:

[
    {
        "datasetGroupName": "edge_response_time",
        "datasetGroupDescription": "Edge Response Time",
        "datasetFields": [
            {
                "datasetFieldId": 86,
                "datasetFieldName": "edge_response_time",
                "datasetFieldDescription": "Edge Response Time"
            }
        ]
    },
    {
        "datasetGroupName": "http_status_cd",
        "datasetGroupDescription": "HTTP Status Code",
        "datasetFields": [
            {
                "datasetFieldId": 77,
                "datasetFieldName": "2xx",
                "datasetFieldDescription": "2xx"
            },
            {
                "datasetFieldId": 78,
                "datasetFieldName": "3xx",
                "datasetFieldDescription": "3xx"
            },
            {
                "datasetFieldId": 79,
                "datasetFieldName": "4xx",
                "datasetFieldDescription": "4xx"
            },
            {
                "datasetFieldId": 80,
                "datasetFieldName": "5xx",
                "datasetFieldDescription": "5xx"
            }
        ]
    },
    {
        "datasetGroupName": "traffic_volumes",
        "datasetGroupDescription": "Traffic Volumes",
        "datasetFields": [
            {
                "datasetFieldId": 81,
                "datasetFieldName": "Requests per second",
                "datasetFieldDescription": "Requests per second"
            },
            {
                "datasetFieldId": 82,
                "datasetFieldName": "Bytes per second",
                "datasetFieldDescription": "Bytes per second"
            }
        ]
    },
    {
        "datasetGroupName": "cdn_offload",
        "datasetGroupDescription": "CDN Offload",
        "datasetFields": [
            {
                "datasetFieldId": 83,
                "datasetFieldName": "Count of cache hits",
                "datasetFieldDescription": "Count of cache hits"
            },
            {
                "datasetFieldId": 84,
                "datasetFieldName": "Count of cache misses",
                "datasetFieldDescription": "Count of cache misses"
            },
            {
                "datasetFieldId": 85,
                "datasetFieldName": "Offload rate",
                "datasetFieldDescription": "Offload rate"
            }
        ]
    },
    {
        "datasetGroupName": "origin_response_time",
        "datasetGroupDescription": "Origin Response Time",
        "datasetFields": [
            {
                "datasetFieldId": 91,
                "datasetFieldName": "origin_response_time",
                "datasetFieldDescription": "Origin Response Time"
            }
        ]
    }
]

List groups

Returns groups within the context of your account that let you view and create data stream configurations in properties that they have access to.

GET /datastream-config-api/v1/datastream1/groups

Status 200 application/json

Object type: Group

Download schema: Groups.json

Response body:

[
    {
        "parentGroupId": 20628,
        "groupId": 84992,
        "groupName": "Example Group1",
        "description": null,
        "accountId": "1-BGSIGR",
        "enabled": true,
        "contractIds": [
            "1-GNSLXD"
        ],
        "childGroupIds": []
    },
    {
        "parentGroupId": 20628,
        "groupId": 59395,
        "groupName": "Sample Group1",
        "description": null,
        "accountId": "1-BGSIGR",
        "enabled": true,
        "contractIds": [
            "1-GNLXD"
        ],
        "childGroupIds": []
    }
]

List streams

Returns the latest versions of the data stream configurations for all groups within in your account. If you provide the groupId as a query parameter, the operation returns the latest versions of the data streams in the group.

GET /datastream-config-api/v1/datastream1/streams{?groupId}

Sample: /datastream-config-api/v1/datastream1/streams?groupId=6483

Parameter Type Sample Description
Optional query parameters
groupId Integer 6483 Identifies the group.

Status 200 application/json

Object type: StreamVersion

Download schema: Streams.json

Response body:

[
    {
        "streamId": 6483,
        "streamName": "Example Stream",
        "streamVersionId": 2,
        "createdBy": "user",
        "createdDate": "23-07-2020 08:01:28 GMT",
        "currentVersionId": 2,
        "archived": false,
        "activationStatus": "ACTIVATED",
        "groupId": 21483,
        "groupName": "Group Name",
        "contractId": "1-FE6JR",
        "streamType": "AGGREGATE",
        "connectors": [],
        "properties": [
            {
                "propertyId": 349772,
                "propertyName": "property.com"
            }
        ]
    },
    {
        "streamId": 4654,
        "streamName": "Example Stream2",
        "streamVersionId": 3,
        "createdBy": "user1",
        "createdDate": "15-07-2020 09:55:25 GMT",
        "currentVersionId": 3,
        "archived": false,
        "activationStatus": "ACTIVATING",
        "groupId": 21483,
        "groupName": "Performance Group-1-FE6JR",
        "contractId": "1-FE6JR",
        "streamType": "AGGREGATE",
        "connectors": [
            "SumoLogic_Connector",
            "Splunk_Connector",
            "Datadog_Connector"
        ],
        "properties": [
            {
                "propertyId": 46337,
                "propertyName": "property-name.com"
            }
        ]
    },
    {
        "streamId": 7755,
        "streamName": "Example Stream3",
        "streamVersionId": 2,
        "createdBy": "user",
        "createdDate": "14-07-2020 11:35:59 GMT",
        "currentVersionId": 2,
        "archived": false,
        "activationStatus": "DEACTIVATING",
        "groupId": 21483,
        "groupName": "Performance Group-1-FE6JR",
        "contractId": "1-FE6JR",
        "streamType": "AGGREGATE",
        "connectors": [
            "Splunk_Connector",
            "S3_Connector"
        ],
        "properties": [
            {
                "propertyId": 46337,
                "propertyName": "property-name.com"
            }
        ]
    },
    {
        "streamId": 2459,
        "streamName": "A Sample Raw Stream",
        "streamVersionId": 4,
        "createdBy": "user",
        "createdDate": "14-07-2020 09:01:14 GMT",
        "currentVersionId": 4,
        "archived": false,
        "activationStatus": "ACTIVATING",
        "groupId": 21483,
        "groupName": "Group Name2",
        "contractId": "1-FE6JR",
        "streamType": "RAW",
        "connectors": [
            "S3_Connector"
        ],
        "properties": [
            {
                "propertyId": 46337,
                "propertyName": "property-name.com"
            },
            {
                "propertyId": 163203,
                "propertyName": "property-name1.com"
            }
        ]
    }
]
  1. If you don’t already have the optional groupId query parameter, run the List groups operation and store the relevant value.

  2. Make a GET request to datastream-config-api/v1/datastream1/streams{?groupId}.

Create a stream

Creates a data stream configuration. Within a stream configuration, you can select properties to monitor in the data stream, data set parameters to deliver in logs, and one or more destinations to send these logs to. You can also configure your stream to store your log data and make it available through the DataStream 1 Pull API. See Stream mode in API concepts.

POST /datastream-config-api/v1/datastream1/streams

Content-Type: application/json

Object type: StreamConfiguration

Download schema: SaveStreamRequest.json

Request body:

{
    "streamName": "Aggregated logs stream",
    "streamType": "AGGREGATE",
    "groupId": 20628,
    "contractId": "1-GNLXD",
    "timeFrame": 15,
    "propertyIds": [
        472787
    ],
    "datasetFieldIds": [
        86,
        79,
        80,
        81,
        82
    ],
    "connectors": [
        {
            "url": "https://endpoint2.collection.us2.sumologic.com/receiver/v1/http/ZaVnC4dhaV3Z6XTKv2zgi6YdjR2a9ZpU7GhuVgWxr3xod1eGTAztg6t_8fLbxakler2gm_l7FepmaQFj-Nc9_GLJItWCJgFkVfFSF75x_LHzVZSHIkQ4xQ==",
            "connectorName": "Https_connector",
            "connectorType": "HTTPS"
        }
    ]
}

Status 202 application/json

Object type: StreamConfiguration

Download schema: SaveStreamUpdate.json

Response body:

{
    "streamVersionKey": {
        "streamId": 6286,
        "streamVersionId": 1
    },
    "affectedStreams": [
        {
            "streamName": "Sample Name",
            "streamVersionKey": {
                "streamId": 6286,
                "streamVersionId": 1
            }
        },
        {
            "streamName": "test stream",
            "streamVersionKey": {
                "streamId": 6285,
                "streamVersionId": 1
            }
        }
    ],
    "affectedProperties": [
        {
            "propertyId": 87202,
            "propertyName": "property.com"
        }
    ]
}
  1. Determine the type of logs that you want the stream to collect, either RAW or AGGREGATE. See Data sets in API concepts.

  2. Get and store these values:

    • Run the List properties operation and store the propertyId values of the properties that you want to monitor in the stream.

    • Run the List data set parameters operation and store datasetFieldId values of the fields that you want the stream to collect.

    • Run the List groups operation and store the groupId and contractId values of the group that you want to use in the stream.

  3. Build one or more connector objects for the connectors array. These are the destinations where you want your stream to deliver logs. You can also send an empty connectors array to store the logs in the system and retrieve them with the DataStream 1 Pull API. These are the connector objects that you can use:

  4. Build a StreamConfiguration object.

  5. POST the object to /datastream-config-api/api/v1/datastream1/streams.

View a stream’s version

Returns a version of a data stream, including details about its monitored properties, logged data set parameters, and delivery destinations. If you omit the version query parameter, this operation returns the latest version of the data stream.

GET /datastream-config-api/v1/datastream1/streams/{streamId}{?version}

Sample: /datastream-config-api/v1/datastream1/streams/6301?version=1

Parameter Type Sample Description
URL path parameters
streamId Integer 6301 Identifies the stream.
Optional query parameters
version Integer 1 Identifies the version of the stream. If omitted, the operation returns the latest version of the data stream.

Status 200 application/json

Object type: StreamVersion

Download schema: StreamDetail.json

Response body:

{
    "streamId": 10187,
    "streamName": "Example Stream",
    "streamVersionId": 23,
    "createdBy": "user",
    "createdDate": "30-07-2020 07:22:40 GMT",
    "activationStatus": "ACTIVATED",
    "groupId": 20628,
    "groupName": "Group Name",
    "contractId": "1-GNLXD",
    "streamType": "AGGREGATE",
    "modifiedBy": "user",
    "modifiedDate": "02-08-2020 08:23:31 GMT",
    "timeFrame": 1,
    "datasets": [
        {
            "datasetGroupName": "edge_response_time",
            "datasetGroupDescription": "Edge Response Time",
            "datasetFields": [
                {
                    "datasetFieldId": 86,
                    "datasetFieldName": "edge_response_time",
                    "datasetFieldDescription": "Edge Response Time"
                }
            ]
        },
        {
            "datasetGroupName": "http_status_cd",
            "datasetGroupDescription": "HTTP Status Code",
            "datasetFields": [
                {
                    "datasetFieldId": 77,
                    "datasetFieldName": "2xx",
                    "datasetFieldDescription": "2xx"
                },
                {
                    "datasetFieldId": 78,
                    "datasetFieldName": "3xx",
                    "datasetFieldDescription": "3xx"
                },
                {
                    "datasetFieldId": 79,
                    "datasetFieldName": "4xx",
                    "datasetFieldDescription": "4xx"
                },
                {
                    "datasetFieldId": 80,
                    "datasetFieldName": "5xx",
                    "datasetFieldDescription": "5xx"
                }
            ]
        },
        {
            "datasetGroupName": "traffic_volumes",
            "datasetGroupDescription": "Traffic Volumes",
            "datasetFields": [
                {
                    "datasetFieldId": 81,
                    "datasetFieldName": "Requests per second",
                    "datasetFieldDescription": "Requests per second"
                },
                {
                    "datasetFieldId": 82,
                    "datasetFieldName": "Bytes per second",
                    "datasetFieldDescription": "Bytes per second"
                }
            ]
        },
        {
            "datasetGroupName": "cdn_offload",
            "datasetGroupDescription": "CDN Offload",
            "datasetFields": [
                {
                    "datasetFieldId": 83,
                    "datasetFieldName": "Count of cache hits",
                    "datasetFieldDescription": "Count of cache hits"
                },
                {
                    "datasetFieldId": 84,
                    "datasetFieldName": "Count of cache misses",
                    "datasetFieldDescription": "Count of cache misses"
                },
                {
                    "datasetFieldId": 85,
                    "datasetFieldName": "Offload rate",
                    "datasetFieldDescription": "Offload rate"
                }
            ]
        },
        {
            "datasetGroupName": "origin_response_time",
            "datasetGroupDescription": "Origin Response Time",
            "datasetFields": [
                {
                    "datasetFieldId": 91,
                    "datasetFieldName": "origin_response_time",
                    "datasetFieldDescription": "Origin Response Time"
                }
            ]
        }
    ],
    "connectors": [
        {
            "connectorId": 11178,
            "connectorType": "DATADOG",
            "connectorName": "DataDog Connector",
            "url": "https://http-intake.logs.datadoghq.com/v1/input/"
        }
    ],
    "properties": [
        {
            "propertyId": 382631,
            "propertyName": "property1.com"
        },
        {
            "propertyId": 347459,
            "propertyName": "property2.com"
        },
        {
            "propertyId": 389150,
            "propertyName": "property3.com"
        }
    ]
}
  1. If you don’t already have the streamId value and the optional query parameter version value, run the List streams operation and store the relevant values.

  2. Get a version of the stream:

    • To get the latest version of the stream, make a GET request to /datastream-config-api/v1/datastream1/streams/{streamId}.

    • To get a specific version of the stream, use the streamVersionId as a version query parameter and make a GET request to /datastream-config-api/v1/datastream1/streams/{streamId}{?version}.

Edit a stream

Updates the latest version of a data stream. Running this operation creates a version a data stream that becomes a default version for this stream. Note that only active streams collect and send logs to their destinations. See Version management and Activate a stream.

PUT /datastream-config-api/v1/datastream1/streams/{streamId}

Sample: /datastream-config-api/v1/datastream1/streams/6301

Content-Type: application/json

Object type: StreamConfiguration

Download schema: EditStreamRequest.json

Request body:

{
    "streamName": "trial",
    "streamType": "RAW",
    "groupId": 158842,
    "contractId": "1-FE6JR",
    "sampleRate": 48,
    "timeFrame": null,
    "propertyIds": [
        180934
    ],
    "datasetFieldIds": [
        65,
        66,
        67,
        68,
        69,
        70,
        71,
        72,
        73,
        74,
        75,
        76,
        98,
        99,
        100,
        101,
        102
    ]
}
Parameter Type Sample Description
URL path parameters
streamId Integer 6301 Identifies the stream.

Status 202 application/json

Object type: StreamConfiguration

Download schema: SaveStreamUpdate.json

Response body:

{
    "streamVersionKey": {
        "streamId": 6286,
        "streamVersionId": 1
    },
    "affectedStreams": [
        {
            "streamName": "Sample Name",
            "streamVersionKey": {
                "streamId": 6286,
                "streamVersionId": 1
            }
        },
        {
            "streamName": "test stream",
            "streamVersionKey": {
                "streamId": 6285,
                "streamVersionId": 1
            }
        }
    ],
    "affectedProperties": [
        {
            "propertyId": 87202,
            "propertyName": "property.com"
        }
    ]
}
  1. Get and store these values:

    • Run the List streams operation and store the relevant streamId value.

    • Run the List properties operation and store the propertyId values of the properties that you want to monitor in the stream.

    • Run the List data set parameters operation and store datasetFieldId values of the fields that you want the stream to collect.

    • Run the List groups operation and store the groupId and contractId values of the group that you want to use in the stream.

  2. Build one or more connector objects for the connectors array. These are the destinations where you want your stream to deliver logs. You can also send an empty connectors array to store the logs in the system and retrieve them with the DataStream 1 Pull API. These are the connector objects that you can use:

  3. Build a StreamConfiguration object.

  4. PUT the object to /datastream-config-api/api/v1/datastream1/streams.

Delete a stream

Deletes a stream. Deleting a stream means that you won’t be able to activate this data stream again, and that you’ll stop receiving logs for the properties that this data stream monitors. To delete a data stream, you need to run the Deactivate a stream operation.

DELETE /datastream-config-api/v1/datastream1/streams/{streamId}

Sample: /datastream-config-api/v1/datastream1/streams/6301

Parameter Type Sample Description
URL path parameters
streamId Integer 6301 Identifies the stream.

Status 200 application/json

Download schema: DeleteStream.json

Response body:

{
    "message": "Success"
}
  1. If you don’t already have the streamId value, run the List streams operation and store the relevant value.

  2. Run the Deactivate a stream operation.

  3. Make a DELETE request to: /datastream-config-api/v1/datastream1/streams/{streamId}.

Activate a stream

Activates the latest version of a data stream. Activating a data stream takes about 15 minutes. Once a data stream is active, it starts collecting and sending logs to its destinations. If you want to stop receiving these logs, you can deactivate a data stream at any time. See the Deactivate a stream operation.

PUT /datastream-config-api/v1/datastream1/streams/{streamId}/activate

Sample: /datastream-config-api/v1/datastream1/streams/6301/activate

Parameter Type Sample Description
URL path parameters
streamId Integer 6301 Identifies the stream.

Status 202 application/json

Object type: StreamUpdate

Download schema: StreamUpdate.json

Response body:

{
    "streamVersionKey": {
        "streamId": 6301,
        "streamVersionId": 1
    }
}
  1. If you don’t already have the streamId value, run the List streams operation and store the relevant value.

  2. Make a PUT request to /datastream-config-api/v1/datastream1/streams/{streamId}/activate.

Deactivate a stream

Deactivates the latest version of a stream. Deactivating a stream means that you stop receiving logs for the properties that this data stream monitors. Deactivating a data stream takes approximately 15 minutes. If you want to start receiving these logs again, you can activate this data stream at any time. See the Activate a stream operation.

PUT /datastream-config-api/v1/datastream1/streams/{streamId}/deactivate

Sample: /datastream-config-api/v1/datastream1/streams/6301/deactivate

Parameter Type Sample Description
URL path parameters
streamId Integer 6301 Identifies the stream.

Status 202 application/json

Object type: StreamUpdate

Download schema: StreamUpdate.json

Response body:

{
    "streamVersionKey": {
        "streamId": 6301,
        "streamVersionId": 1
    }
}
  1. If you don’t already have the streamId value, run the List streams operation and store the relevant value.

  2. Make a PUT request to /datastream-config-api/v1/datastream1/streams/{streamId}/deactivate.

Data

This section provides you with the data model for the DataStream 1 Configuration API.

Download the JSON schemas for this API.

This section’s data schema tables list membership requirements as follows:

Member is required in requests, or always present in responses, even if its value is empty or null.
Member is optional, and may be omitted in some cases.
Member is out of scope, and irrelevant to the specified interaction context. If you include the member in that context, it either triggers an error, or is ignored.

Property

Provides details about a property within the context of the group.

Download schema: Property.json

Sample GET response:

{
    "propertyId": 382631,
    "propertyName": "property.com"
}

Property members

Member Type Required Description
Property: Provides details about a property within the context of the group.
propertyId Integer The identifier of the property.
propertyName String The descriptive label for the property.

Group

Provides detailed information about the hierarchical context of the group. It also informs you whether you can view and create data stream configurations within this group.

Download schema: Group.json

Sample GET response:

{
    "parentGroupId": 20628,
    "groupId": 49851,
    "groupName": "Example Group Name",
    "description": null,
    "accountId": "1-BGIGR",
    "enabled": true,
    "contractIds": [
        "1-GNLXD"
    ],
    "childGroupIds": [
        98579,
        53491,
        130231,
        58588
    ]
}

Group members

Member Type Required Description
Group: Provides detailed information about the hierarchical context of the group. It also informs you whether you can view and create data stream configurations within this group.
accountId String Identifies the account that the group is part of.
childGroupIds Array Identifies the child groups within the group.
contractIds Array Identifies the contracts associated with the group.
description String, Null Describes the group.
enabled Boolean Whether this group allows you to create and view data stream configurations.
groupId Integer Identifies the group.
groupName String Identifies the name of the group.
parentGroupId Integer Identifies the parent group.

Dataset

Provides information about a group of data set parameters available.

Download schema: Dataset.json

Sample GET response:

{
    "datasetGroupName": "network",
    "datasetGroupDescription": "Network Data",
    "datasetFields": [
        {
            "datasetFieldId": 72,
            "datasetFieldName": "nw",
            "datasetFieldDescription": "Network"
        },
        {
            "datasetFieldId": 73,
            "datasetFieldName": "nwType",
            "datasetFieldDescription": "Network Type"
        },
        {
            "datasetFieldId": 74,
            "datasetFieldName": "throughput",
            "datasetFieldDescription": "Throughput"
        },
        {
            "datasetFieldId": 75,
            "datasetFieldName": "bw",
            "datasetFieldDescription": "Bandwidth"
        },
        {
            "datasetFieldId": 76,
            "datasetFieldName": "proxy",
            "datasetFieldDescription": "Proxy"
        }
    ]
}

Dataset members

Member Type Description
Dataset: Provides information about a group of data set parameters available.
datasetFields Dataset.datasetFields[] A list of data set parameters available within the data set group.
datasetGroupDescription String Describes the dataset group.
datasetGroupName String A name of the dataset group.
Dataset.datasetFields[]: A list of data set parameters available within the data set group.
datasetFieldDescription String Describes the data set field.
datasetFieldId Integer Identifies the field.
datasetFieldName String A name of the data set field.

StreamVersion

Provides detailed information about the latest configuration version of a data stream. It provides selected data set parameters, associated property identifiers, destination details, and version status.

Download schema: StreamDetail.json

Sample GET response:

{
    "streamId": 10187,
    "streamName": "Example Stream",
    "streamVersionId": 23,
    "createdBy": "user",
    "createdDate": "30-07-2020 07:22:40 GMT",
    "activationStatus": "ACTIVATED",
    "groupId": 20628,
    "groupName": "Group Name",
    "contractId": "1-GNLXD",
    "streamType": "AGGREGATE",
    "modifiedBy": "user",
    "modifiedDate": "02-08-2020 08:23:31 GMT",
    "timeFrame": 1,
    "datasets": [
        {
            "datasetGroupName": "edge_response_time",
            "datasetGroupDescription": "Edge Response Time",
            "datasetFields": [
                {
                    "datasetFieldId": 86,
                    "datasetFieldName": "edge_response_time",
                    "datasetFieldDescription": "Edge Response Time"
                }
            ]
        },
        {
            "datasetGroupName": "http_status_cd",
            "datasetGroupDescription": "HTTP Status Code",
            "datasetFields": [
                {
                    "datasetFieldId": 77,
                    "datasetFieldName": "2xx",
                    "datasetFieldDescription": "2xx"
                },
                {
                    "datasetFieldId": 78,
                    "datasetFieldName": "3xx",
                    "datasetFieldDescription": "3xx"
                },
                {
                    "datasetFieldId": 79,
                    "datasetFieldName": "4xx",
                    "datasetFieldDescription": "4xx"
                },
                {
                    "datasetFieldId": 80,
                    "datasetFieldName": "5xx",
                    "datasetFieldDescription": "5xx"
                }
            ]
        },
        {
            "datasetGroupName": "traffic_volumes",
            "datasetGroupDescription": "Traffic Volumes",
            "datasetFields": [
                {
                    "datasetFieldId": 81,
                    "datasetFieldName": "Requests per second",
                    "datasetFieldDescription": "Requests per second"
                },
                {
                    "datasetFieldId": 82,
                    "datasetFieldName": "Bytes per second",
                    "datasetFieldDescription": "Bytes per second"
                }
            ]
        },
        {
            "datasetGroupName": "cdn_offload",
            "datasetGroupDescription": "CDN Offload",
            "datasetFields": [
                {
                    "datasetFieldId": 83,
                    "datasetFieldName": "Count of cache hits",
                    "datasetFieldDescription": "Count of cache hits"
                },
                {
                    "datasetFieldId": 84,
                    "datasetFieldName": "Count of cache misses",
                    "datasetFieldDescription": "Count of cache misses"
                },
                {
                    "datasetFieldId": 85,
                    "datasetFieldName": "Offload rate",
                    "datasetFieldDescription": "Offload rate"
                }
            ]
        },
        {
            "datasetGroupName": "origin_response_time",
            "datasetGroupDescription": "Origin Response Time",
            "datasetFields": [
                {
                    "datasetFieldId": 91,
                    "datasetFieldName": "origin_response_time",
                    "datasetFieldDescription": "Origin Response Time"
                }
            ]
        }
    ],
    "connectors": [
        {
            "connectorId": 11178,
            "connectorType": "DATADOG",
            "connectorName": "DataDog Connector",
            "url": "https://http-intake.logs.datadoghq.com/v1/input/"
        }
    ],
    "properties": [
        {
            "propertyId": 382631,
            "propertyName": "property1.com"
        },
        {
            "propertyId": 347459,
            "propertyName": "property2.com"
        },
        {
            "propertyId": 389150,
            "propertyName": "property3.com"
        }
    ]
}

StreamVersion members

Member Type Description
StreamVersion: Provides detailed information about the latest configuration version of a data stream. It provides selected data set parameters, associated property identifiers, destination details, and version status.
activationStatus Enumeration The activation status of the data stream configuration version. These are possible values: ACTIVATING, ACTIVATED, DEACTIVATING, or DEACTIVATED. See the Activate a stream and Deactivate a stream operations.
connectors Array of these objects: Amazon S3, Https, Splunk, SumoLogic, or Datadog Provides detailed information about connector configurations in the stream when viewing a stream’s version or the connector names when listing available streams. If this array is empty, it means that the stream doesn’t send logs to any destination, but it can access them with the DataStream 1 Pull API.
contractId String Identifies the contract that you created the stream for.
createdBy String The username who created the stream.
createdDate String The date and time when the stream was created.
datasets StreamVersion.datasets[] A list of data set parameters selected from the associated template that the stream monitors in logs.
groupId Integer Identifies the group that has access to the product and that you created the stream configuration for.
groupName String The name of the user group that you created the stream for.
modifiedBy String The username who modified the stream.
modifiedDate String The date and time when the stream was modified.
properties StreamVersion.properties[] Identifies the properties that you monitor in the stream.
streamId Integer Identifies the stream.
streamName String The name of the stream.
streamType Enumeration Specifies the type of stream, either RAW or AGGREGATE.
streamVersionId Integer Identifies the configuration version of the stream.
StreamVersion.datasets[]: A list of data set parameters selected from the associated template that the stream monitors in logs.
datasetFields StreamVersion.datasets[].datasetFields[] A list of data set parameters in the group that the stream monitors.
datasetGroupDescription String A descriptive label for the dataset group.
datasetGroupName String The name of the dataset group.
StreamVersion.datasets[].datasetFields[]: A list of data set parameters in the group that the stream monitors.
datasetFieldDescription String Describes the data set field.
datasetFieldId Integer Identifies the field.
datasetFieldName String A name of the data set field.
StreamVersion.properties[]: Identifies the properties that you monitor in the stream.
propertyId Integer The identifier of the property.
propertyName String The descriptive label for the property.

StreamConfiguration

Provides information that you need to specify when creating a data stream configuration.

Download schema: SaveStreamRequest.json, EditStreamRequest.json

Sample POST request:

{
    "streamName": "Aggregated logs stream",
    "streamType": "AGGREGATE",
    "groupId": 20628,
    "contractId": "1-GNLXD",
    "timeFrame": 15,
    "propertyIds": [
        472787
    ],
    "datasetFieldIds": [
        86,
        79,
        80,
        81,
        82
    ],
    "connectors": [
        {
            "url": "https://endpoint2.collection.us2.sumologic.com/receiver/v1/http/ZaVnC4dhaV3Z6XTKv2zgi6YdjR2a9ZpU7GhuVgWxr3xod1eGTAztg6t_8fLbxakler2gm_l7FepmaQFj-Nc9_GLJItWCJgFkVfFSF75x_LHzVZSHIkQ4xQ==",
            "connectorName": "Https_connector",
            "connectorType": "HTTPS"
        }
    ]
}

StreamConfiguration members

Member Type POST PUT Description
StreamConfiguration: Provides information that you need to specify when creating a data stream configuration.
connectors Array of these objects: Amazon S3, Https, Splunk, SumoLogic, or Datadog The connector configuration in the data stream.
contractId String Identifies the contract.
datasetFieldIds Array Identifiers of the data set parameters that you want to monitor in logs.
groupId Integer Identifies the group that has access to this data stream’s configuration.
propertyIds Array Identifies the properties that you want to monitor in the stream. Note that a stream can only log data for active properties. You can activate a property in Property Manager in Property Manager.
sampleRate Integer Specifies the sampling percentage of data to send to your destination for the RAW stream type. By default, it’s 25.
streamName String The name of the stream.
streamType Enumeration The type of stream that you want to create, either RAW or AGGREGATE.
timeFrame String Specifies how often you want the system to send the log file to your destination for the AGGREGATE stream type. These are possible values: 1, 5, 15, 30, or 60 minutes. By default, it’s 1.

StreamUpdate

Provides information about a data stream that you created or edited. It also provides information about streams that share properties with the created or edited configuration.

Download schema: SaveStreamUpdate.json

Sample POST response:

{
    "streamVersionKey": {
        "streamId": 6286,
        "streamVersionId": 1
    },
    "affectedStreams": [
        {
            "streamName": "Sample Name",
            "streamVersionKey": {
                "streamId": 6286,
                "streamVersionId": 1
            }
        },
        {
            "streamName": "test stream",
            "streamVersionKey": {
                "streamId": 6285,
                "streamVersionId": 1
            }
        }
    ],
    "affectedProperties": [
        {
            "propertyId": 87202,
            "propertyName": "property.com"
        }
    ]
}

StreamUpdate members

Member Type Description
StreamUpdate: Provides information about a data stream that you created or edited. It also provides information about streams that share properties with the created or edited configuration.
affectedProperties StreamUpdate.affectedProperties[] The set of properties that the affectedStreams share with the created or edited configuration.
affectedStreams StreamUpdate.affectedStreams[] The set of streams that share properties with the created or edited configuration.
streamVersionKey StreamUpdate.streamVersionKey Provides the stream and version identifiers of the created or edited configuration.
StreamUpdate.affectedProperties[]: The set of properties that the affectedStreams share with the created or edited configuration.
propertyId Integer Identifies the shared property.
propertyName String Identifies the property’s name.
StreamUpdate.affectedStreams[]: The set of streams that share properties with the created or edited configuration.
streamName String Identifies the stream’s name.
streamVersionKey StreamUpdate.affectedStreams[].streamVersionKey Provides the stream and version identifiers of the configuration that shares properties with the created or edited configuration.
StreamUpdate.affectedStreams[].streamVersionKey: Provides the stream and version identifiers of the configuration that shares properties with the created or edited configuration.
streamId Integer Identifies the stream that shares properties with the created or edited configuration.
streamVersionId Integer Identifies the latest version of the stream.
StreamUpdate.streamVersionKey: Provides the stream and version identifiers of the created or edited configuration.
streamId Integer Identifies the stream that you created or edited.
streamVersionId Integer Identifies the latest version of the stream that you created or edited.

AmazonS3

Provides details about the Amazon S3 connector in a data stream.

Download schema: S3SavedConnector.json, S3.json

Sample POST request:

{
    "path": "log",
    "connectorName": "S3_connector",
    "bucket": "media-datastream.akamai.com",
    "region": "AP_Mumbai",
    "accessKey": "AKIA6DK7TDQLVGZ3TYP1",
    "secretAccessKey": "1T2ll1H4dXWx5itGhpc7FlSbvvOvky1098nTtEMg",
    "connectorType": "S3"
}

AmazonS3 members

Member Type Required Description
AmazonS3: Provides details about the Amazon S3 connector in a data stream.
accessKey String The access key identifier that you use to authenticate requests to your S3 account. See Managing access keys in AWS API.
bucket String The name of the S3 bucket. See Working with Amazon S3 buckets in AWS.
connectorId Integer Identifies the connector associated with the data stream.
connectorName String The name of the connector.
connectorType Enumeration The name of the connector type. Set it to S3 for this connector type.
path String The folder path associated with your S3 bucket where you want to store your logs.
region String The AWS region where your S3 bucket resides. See Regions, availability zones, and local zones in AWS.
secretAccessKey String The secret access key identifier that you use to authenticate requests to your S3 account.

Datadog

Provides details about the Datadog connector configuration in a data stream.

Download schema: DatadogSavedConnector.json, Datadog.json

Sample POST request:

{
    "service": "datastream-config-api",
    "authToken": "6fe69bf3791009bd7b3bv18dc2c68fe",
    "connectorName": "Datadog_connector",
    "url": "https://http-intake.logs.datadoghq.com/v1/input/",
    "connectorType": "DATADOG"
}

Datadog members

Member Type Required Description
Datadog: Provides details about the Datadog connector configuration in a data stream.
authToken String The API key associated with your Datadog account. See View API keys in Datadog.
connectorId Integer Identifies the connector associated with the data stream.
connectorName String The name of the connector.
connectorType Enumeration The name of the connector type. Set it to DATADOG for this connector type.
service String The service of the Datadog connector. See View Datadog services list.
url String The Datadog URL where you want to store logs.

Splunk

Provides detailed information about Splunk connector that you can use in your stream.

Download schema: SplunkSavedConnector.json, Splunk.json

Sample POST request:

{
    "connectorName": "Splunk_connector",
    "url": "https://http-intake.logs.splunk.com/v1/input/",
    "authToken": "244fd617-51c5-4b2e-916d-54fb-af62",
    "connectorType": "SPLUNK"
}

Splunk members

Member Type Required Description
Splunk: Provides detailed information about Splunk connector that you can use in your stream.
authToken String The API key associated with your Splunk account. See View usage of Auth tokens in Splunk.
connectorId Integer Identifies the connector associated with the data stream.
connectorName String The name of the connector.
connectorType Enumeration The name of the connector type. Set this value to SPLUNK for this connector type.
url String The Splunk URL where you want to store your logs.

SumoLogic

Provides detailed information about a Sumo Logic connector that you can use in your stream.

Download schema: SumoLogicSavedConnector.json, SumoLogic.json

Sample POST request:

{
    "connectorName": "Sumo_connector",
    "url": "https://endpoint2.collection.us2.sumologic.com/receiver/v1/http/ZaVnC4dhaV3Z6XTKv2zgi6YdjR2a9ZpU7GhuVgWxr3xod1eGTAztg6t_8fLbxakler2gm_l7FepmaQFj-Nc9_GLJItWCJgFkVfFSF75x_LHzVZSHIkQ4xQ==",
    "connectorType": "SUMO_LOGIC"
}

SumoLogic members

Member Type Required Description
SumoLogic: Provides detailed information about a Sumo Logic connector that you can use in your stream.
connectorId Integer Identifies the connector associated with the data stream.
connectorName String The name of the connector.
connectorType Enumeration The name of the connector type. Set this value to SUMO_LOGIC for this connector type.
url String The Sumo Logic URL where you want to store your logs.

Https

Provides detailed information about a custom HTTPS connector that you can use in your stream.

Download schema: HTTPSSavedConnector.json, HTTPS.json

Sample POST request:

{
    "connectorName": "Https_connector",
    "url": "https://endpoint2.collection.us2.sumologic.com/receiver/v1/http/ZaVnC4dhaV3Z6XTKv2zgi6YdjR2a9ZpU7GhuVgWxr3xod1eGTAztg6t_8fLbxakler2gm_l7FepmaQFj-Nc9_GLJItWCJgFkVfFSF75x_LHzVZSHIkQ4xQ==",
    "connectorType": "HTTPS"
}

Https members

Member Type Required Description
Https: Provides detailed information about a custom HTTPS connector that you can use in your stream.
connectorId Integer Identifies the connector associated with the data stream.
connectorName String The name of the connector.
connectorType Enumeration The name of the connector type. Set this value to HTTPS for this connector type.
url String The secure URL where you want to store your logs.

Errors

This section provides details on the data object that reflect the API’s common response to error cases, and lists the API’s range of response status codes for both error and success cases.

Error responses

In error cases, the API responds with JSON objects that follow the HTTP Problem Details. This sample shows a bad request error, where the title is a descriptive label for the overall problem, and the instance may be useful if you need to communicate the problem to your Akamai support representative. It also includes an optional errors array that lists potentially more than one problem detected in the request.

{
  "type": "bad-request",
  "title": "Erroneous data input",
  "instance": "8ed959ae-bc22-43f4-893c-2f293518f258",
  "status": 400,
  "errors": [
    {
      "type": "bad-request",
      "title": "Bad Request",
      "instance": "1664e1a7-d916-4cf9-944f-2d9f6d176f8f",
      "detail": "Expiry Date of Previous Key is required"
    }
  ]
}

HTTP status codes

The API produces these set of HTTP status codes for both success and failure scenarios:

Code Description
200 The operation succeeded.
201 Resource created.
202 Resource successfully accepted. This is returned on an activation request. It doesn’t mean that the activation succeeded, but that the API was able to act upon the request.
400 Bad Request.
401 Unauthorized request.
402 Failed request.
403 Access is forbidden. You don’t have access to the requested resource.
404 Resource not found.
405 Method not allowed.
415 Unsupported media type.
422 Unprocessable entity.
429 Too many requests. See Rate limiting.
500 Internal server error. Unexpected error.