Get Unmatched Real-Time Visibility With DataStream

by Manish Kumar
2520

This is one in a series of posts following up on our announcements from the Developer Zone at Edge 2017. For a complete review of Edge, see our recap page.

We received lots of great feedback at Edge this year about Akamai’s real-time visibility offerings. I’d like to give you a bit more info on one particular product we showed: It’s called DataStream, and it’s a breakthrough technology that gives you unprecedented access to the website and application metrics you need, along with superior control to act on those metrics.

DataStream is a real-time, cloud-based technology that can integrate Web performance data directly into your existing reporting environment using APIs that do pull or push. The diagram below illustrates DataStream’s unique process of data collection and aggregation.

DataStream simplifies the task of monitoring complex Web applications, including those that are deployed to other cloud hosting providers and those that leverage third-party services. It also delivers real-time access to granular levels of information about all application activity, including metrics on complete request-response cycles as well as origin response times.

One specific example of DataStream’s simplicity: We’ve made it easy to select the data sets you want to monitor with a simple check-box interaction. Take a look:

DataStream delivers your monitoring data via an API in a standardized format, with the option of using pre-built connectors for Splunk, Sumo-Logic, NetStorage and Amazon S3 bucket to enable quick and simple integration into any existing monitoring infrastructure.

Configuring DataStream is easy via Akamai’s Luna Control Center; it does not require modifications to your origin application code, deployment of origin networking equipment, or any end-user device instrumentation. That means there’s nothing to maintain or update when existing applications change or when new applications move to the cloud.

Here are five key use cases for DataStream:

1. Real-time access to data feeds with low latency

  • Get access to data within 10 seconds of an event occurrence
  • React quickly to any outage

2. Measure your CDN behavior in real time and get visibility into your “middle mile”

  • Use aggregated data to measure basic usage, performance, errors, and more

3. Teams using DevOps in your organization

  • Measure the recent deployment impact on performance
  • Quickly determine whether config deployments are acceptable or not

4. Seamless access to data using Push APIs and Pull APIs

  • View aggregated and raw data using Pull or Push APIs
  • View data from the previous 24 hours using Pull APIs

5. Data governance

  • Get superior control over your log fields, and stream what you want

Here are some of the aggregated metrics you can get through DataStream:

Traffic volumes
  1. Requests per second to edge
  2. Bytes per second from edge
CDN Offload
  1. Count of requests that were a cache hit
  2. Count of requests that were a cache miss
  3. Offload rate: (cache hits) / ( total requests) over the period
HTTP Status Codes
  1. Count of requests that resulted in 2xx
  2. Count of requests that resulted in 3xx
  3. Count of requests that resulted in 4xx
  4. Count of requests that resulted in 5xx
Edge Response Time
  1. a cache-hit at Akamai
  2. a cache-miss at Akamai
  3. a cache-hit
    • at edge (child) level
    • at a parent level
  4. a cache-miss
    • at edge (child) level
    • at a parent level
  5. typical latency observed for non-cacheable requests
Origin Response Time Typical latency observed between when Akamai requests an object from origin, and when it is returned to Akamai

Here’s the simple, four-step process you can follow to get real-time visibility with DataStream:

1. Select your data:

API Method Type Description
<DataStreamURL>/app/datasets/raw/mapping GET Get the list of all the raw datasets groups with their sub fields information.
<DataStreamURL>/app/datasets/aggregate/mapping GET Get the list of all the aggregate datasets groups with their sub fields information.

 

2. Define your connectors:

API Method Type Description
<DataStreamURL>/app/connectors/type/all GET Get the list of all the raw datasets groups with their sub fields information.

 

3. Save your config and create your stream:

API Method Type Description
<DataStreamURL>/app/streams/save POST Save the stream version details provided in the post body.

 

4. Access your data:

API Method Type Description
<DataStreamURL>/datastream-pull-api/v1/streams/{streamId}/raw-logs{?start,end,page,size} GET Pull the raw logs.
<DataStreamURL>/datastream-pull-api/v1/streams/{streamId}/aggregate-logs{?start,end,FieldName,aggregate,page,size} GET Pull aggregated data.

 

To learn more, watch the “Getting Real-Time Middle-Mile Visibility in Your CDN Behavior with DataStream” presentation, recorded in the Developer Zone at Edge 2017.

DataStream is currently in beta; contact your account team for details.

Categories: DevOps, Edge '17 Highlights

Suggested Article