Get answers to the most common questions regarding DataStream.
Q. What is the DataStream output data format?
The output format for DataStream is JSON. Sample schema and datasets can be downloaded here.
Q. When would I use raw logs versus aggregated metrics?
You can start with aggregated metrics for a snapshot of CDN health. If you see any anomalies in the aggregated metrics, you can start streaming raw logs for root cause analysis or diagnostics. For example, if you’re seeing a large number of 4xx errors over a certain period of time, raw logs can help to understand the cause and potential fixes.
Also, you can turn raw log streams on before and after a new deployment, and then turn them off once the deployment is stable. This helps avoid billing when logs are not needed.
Q. How many hostnames can I simultaneously monitor in a single stream?
You can choose to have multiple hostnames in a single stream. There is no limit on adding hostnames. However, as a best practice, it is recommended to have not more than two or three hostnames per stream for ease of data management and root cause analysis.
Q. Which connectors does DataStream currently support?
DataStream comes with out-of-the-box, push-based connectors for Splunk, Sumo Logic and Amazon S3. See this section for more on DataStream's integrations with popular connectors.
Q. Can I design my own connectors if I want to use the same SIEM tool I’m using for security logs and events?
Yes. If you use a different analytics platform than the ones supported with the out-of-the-box connectors, you can use this Open source API definition to define your own custom pull-based connectors.
Q. We don’t use a log analytics tool. Is there any way we could still use DataStream logs (raw or aggregated)?
You will need a tool to parse and visualize DataStream output. For example, you could choose open source stacks like Grafana or ELK for data parsing and visualization in human-readable formats.
Q. What are the available aggregation time frames?
The options to choose for an aggregation window are 5 minutes, 15 minutes, 30 minutes, and 1 hour. The aggregated data is retained for 24 hours.
Q. What are the latest availability timelines for DataStream?
DataStream is currently in a closed beta, and here is the upcoming roadmap (dates are subject to change):
- Limited Availability: Q1 2019
- General Availability: Q2 2019
Q. Is DataStream still open for beta enrollment?
Due to overwhelming subscription, the beta is currently closed. We have stopped accepting new requests, but please feel free to reach out to your account representative to discuss your interest or next steps.
Q. How will I be able to start using DataStream once it becomes generally available?
DataStream will be made available via the Akamai Marketplace and will be easy to configure via the Akamai Luna Control Center. DataStream will not require any modifications to your application code, any origin networking equipment, or end-user device instrumentation.
DataStream will also have the ability to be accessed via API; this will offer a programmatic alternative to many of the features available in the Luna Control Center. Visit here for more details on the DataStream API.
Q. How is DataStream different from Akamai Cloud Monitor or Log Delivery Service (LDS) that we already use?
The new features that make DataStream Akamai’s next-generation log delivery product compared to LDS and Cloud Monitor are:
- Data retention
- Data aggregation
- Pull APIs (in addition to traditional push mechanism)
- Lower latency
- Ability to define a stream containing only the chosen datasets (simply check/uncheck)
- Ability to turn streams on and off as needed
Q. Does DataStream support security event logs too?
No. DataStream is a log delivery product for transactional events and associated metrics. SIEM Integration is the product that delivers security logs.