SIEM CEF connector

About the sample CEF connector

The sample CEF connector receives security events in near real time using the SIEM OPEN API and converts them from JSON into CEF format. The connector pulls from one single API endpoint based on a pull rate you can configure and processes security events generated from security policies within multiple security configurations.

You can configure the connector to save security events locally in addition to forwarding to a destination host over UDP or TCP using Syslog protocol. For Syslog, the connector leverages CEF format. CEF is an open messaging standard introduced by ArcSight, Inc.

CEF connector diagram

The application outputs data over the Syslog protocol using CEF data format in order to provide a flexible solution that meets the requirements of various SIEM tools.

CEF-Syslog works with HP ArcSight and other SIEM solutions that support syslog/CEF format.

Install CEF connector

System requirements

Akamai’s CEF Connector requires Sun JRE 1.8+ to be installed. The latest JRE can be downloaded from the Sun Java site (Java Platform, Standard Edition) or installed from a software distribution package on Linux.

Verify Java version (Linux): To verify that your JRE is installed and available, use the command java -version

    Hardware requirements

    This application is designed to run on a Linux server with at least:

    • 2 CPU cores
    • 6GB RAM
    • 2GB Free Disk Space
    • Run a Linux Kernel greater than 2.6

    Proxy server

    To access the SIEM API from behind a proxy server, ensure that your proxy:

    • whitelists the domains *.cloudsecurity.akamaiapis.net
    • does not interfere with HTTP request headers for those domains. If, due to a strict enterprise security policy, your proxy does change these headers, make sure that at a minimum you allow and don't change the Host and Authorization headers.

    Install

    Visit https://developer.akamai.com/tools/siem-integration to get the latest CEFConnector distribution package. Transfer the package using either Linux commandwget http://[server]/CEFConnector-1.0.zip (replace [server] with your specific server name) or using SFTP (SSH File Transfer Protocol).

    Unzip the distribution package anywhere on the file system. You can install Unzip from a software distribution package on Linux (for example yum install unzip ).

    To install a service, create a symbolic link to bin/AkamaiCEFConnector.sh shell script in /etc/init.d . You can execute the shell script with the following commands: start | stop | status | resetdb Resetdb deletes cefconnector.db which contains the last successful offset data pull. Removing the file causes the connector to process offset=NULL as long as timebased setting is false . If timebased is true, a new offset is saved after the first successful pull. Learn more about offset and time-based pulls.

    Configure your SIEM event data pull

    Set up and manage your connector through the source pull configuration settings located in the config/CEFConnector.properties file.

    • connector.refresh.period
      • Default Value: 300
      • Description: Defines the pull rate from the Akamai source in seconds.
      • Allowed Values:Positive integer value greater than 0. No decimals allowed. Unallowed values will default.
      • Example: 60 (1 minute)
      • Required: Yes
    • akamai.data.requesturlhost
      • Description: Akamai web service URL (example https://cloudsecurity.akamaiapis.net) This along with the security configurations ids will be used to generate the request url (https://cloudsecurity.akamaiapis.net/siem/v1/configs/14227?offset=NULL).
      • Allowed Values: This value cannot be blank or commented out.
      • Example:https://cloudsecurity.akamaiapis.net
      • Required: Yes
    • akamai.data.configs
      • Description: Add multiple security configuration IDs by separating them with a semicolon";".
      • Example: 12345:67890
      • Required: Yes
    • akamai.data.timebased
      • Description: Set to false to pull security events continuously by offset token (this should be the usual, everyday setting) and set to true in order to pull security events that fall within a specific time frame. If timebased is set to true , the from parameter (see next configuration entry) is required and to is optional. Learn more about retrieving past events.
      • Allowed Values: Positive integer value greater than 0. No decimals allowed. Unallowed values will default.
      • Allowed Values: true or false
      • Example: false
      • Required: Yes
    • akamai.data.timebased.from
      • Description: If timebased parameter is true, this a required field. Enter the time (in epoch format) to start pulling past security events. This can be any time within the 12 hours preceding the present moment. If regular offset event collection occurred within the time window, you may see duplicate data in your SIEM software.
      • Allowed Values: Positive integer in epoch format. Value has to be less than timebased.to .
      • Example: 1491588763
      • Required: Yes, if timebased parameter is true , No, if timebased parameter is false .
    • akamai.data.timebased.to
      • Default Value: No value
      • Description: If timebased parameter is true, you can optionally enter the end timestamp (in epoch format) to a set period for which to pull past security events. If no value or invalid format is provided, default value is used.
      • Allowed Values: Positive integer in epoch format. Value must be greater than or equal to the value in timebased.from .
      • Example: 1491588763
      • Required: No
    • akamai.data.limit
      • Default Value: No value
      • Description: Limit the number of security events to pull. Use this field to prevent overload and protect your SIEM system if it’s suddenly flooded with event data. If no value is provided or an invalid value is provided, the default limit defined by the SIEM OPEN API side is used.
      • Allowed Values: Positive integer values. No decimals allowed. Unallowed values will default.
      • Example: 100
      • Required: No
    • akamai.data.accesstoken
      • Description: Enter the access token you copied when you provisioned the SIEM OPEN API in Akamai’s LUNA portal.
      • Allowed Values: This value cannot be blank or commented out.
      • Example: akab-fc7lrkvv57lgxjpx-mizhsadsasdasym
      • Required: Yes
    • akamai.data.clienttoken
      • Description: Enter the client token you copied when you provisioned the SIEM OPEN API in Akamai’s LUNA portal.
      • Allowed Values: This value cannot be blank or commented out.
      • Example: akab-fc7lrkvv57lgxjpx-mizhsadsasdasym
      • Required: Yes
    • akamai.data.clientsecret
      • Description: Enter the client secret you copied when you provisioned the SIEM OPEN API in Akamai’s LUNA portal.
      • Allowed Values:This value cannot be blank or commented out.
      • Example: LWcGK6h2121GdfdDSD8m+4wllsdfdsG8wgFS+dfDfZQ=
      • Required: Yes
    • akamai.data.baseurl
      • Description: Enter the URL you copied when you provisioned the SIEM OPEN API in Akamai’s LUNA portal.
      • Allowed Values: This value cannot be blank or commented out.
      • Example: akab-dsfdsdypmwkj7a-eqkdfgfdsswsfaoclec.cloudsecurity.akamaiapis.net
      • Required: Yes
    • akamai.cefformatheader
      • Description: CEF Header Values are separated by . If is part of a static string, then it must be escaped with . Values can be static or
      • generated from available functions: requestURL(), eventClassId(), name(), severity(), appliedAction(),ipv6src()
      • Allowed Values: This value can not be blank or commented out.
      • Example: CEF:0|Akamai|akamai_siem|1.0|eventClassId()|name()|severity()
      • Required: Yes
    • akamai.cefformatextension
      • Description: CEF Extension Values are separated by a space. Values can be static, generated from available functions ( eventClassId() name()severity() appliedAction() ipv6src() ) or pulled from JSON Api. JSON api is defined by ${} and each json object is separated by a period. Static Values are defined by quotation marks. Function-generated values are defined by () and must be one of the available functions defined in documentation.
      • Allowed Values: This value can not be blank or commented out.
      • Example: act=appliedAction() app=${httpMessage.protocol}c6a2=ipv6src()c6a2Label="Source IPv6 Address"
      • cs1=${attackData.rules} cs1Label="Rules"
      • cs2=${attackData.ruleMessages} cs2Label="Rule Messages"
      • cs3=${attackData.ruleData} cs3Label="Rule Data"
      • cs4=${attackData.ruleSelectors} cs4Label="Rule Selectors"
      • Required: No
    • akamai.base64fields
      • Description: If an API json object is base64 encoded, it must be defined here.
      • Example: ${attackData.ruleVersions};${attackData.rules};${attackData.ruleActions}
      • Required: No
    • akamai.urlencoded
      • Description: If an API json object is URL-encoded, it must be defined here.
      • Example: ${attackData.ruleVersions};${attackData.rules};${attackData.ruleActions}
      • Required: No
    • akamai.multivaluedelim
      • Default Value: , (comma)
      • Description: Delimiter used to separate multi valued CEF fields Or commented out.
      • Allowed Values: " " is treated the same as "" and a blank will default.
      • Example:, (comma), \u0020 (white space), \n (newline), \t (tab)
      • Required: No
    • connector.consumer.count
      • Default Value: 3
      • Description: Limit the number of consumer threads
      • Example: 10
      • Required: No

    Set up in Arcsight

    1. Log in to Arcsight with administrator rights.
    2. From the menu choose Configuration > Data > Receivers.
    3. Add a new TCP CEF Receiver with a distinct name like Akamai CEF Data. By default, this assigns the receiver to listen on all IP/HOST, port 546 (if not already in use), UTF-8 encoding, and source type CEF. The receiver is enabled by default. The port can be altered if needed.
    4. To group security events by device, go to the menu and choose Configuration > Data > Device Groups.
    5. Add a new Device Group and pick from a list of auto-populated devices. If the connector is installed locally and is sending logs to localhost, the device is Logger Internal Event Device with the receiver name you added in Step 3.
    6. To view the device group security logs, go to the menu and choose Analyze > Search. In the search bar enter _deviceGroup in ["<device group name>"] . Example: _deviceGroup in ["Akamai SIEM -On Server Connector"]

    Logging

    You can set log configurations in the config/log4j2.xml file.

    • log-path: Path location of file logs. Use relative or specific path (example: logs).
    • log-name: File name for logs (example: filename).
    • SizeBasedTriggeringPolicy: Log size rollover limit (example: 1MB).
    • DefaultRolloverStrategy: Max number of log files stored for each log (info, warn, and error) on disk (example: 20).
    • CEFHost: Remote CEF Syslog Server Host (example: 127.0.0.1).
    • CEFPort: Remote CEF Syslog Server Port (example: 514).
    • CEFProtocol: Remote CEF Syslog Server Protocol (example: UDP/TCP).

    To run in debug mode:

    1. In the log4j2.xml file, scroll down to the bottom and under <Loggers> find <Root level="info"
    2. Change info to debug and save the file.

    Find logs in the /bin/logs/cefconnector.log file.

    SIEM API data format for CEF

    The security events data available from the Akamai endpoint arrives in JSON format. Each line is a separate JSON object representing a single request.

    Calculated fields

    $appliedAction
    If attackData.ruleActions contains any action other than 'alert' or 'deny', use it (there will be only one) otherwise, check if there is a 'deny' action and if so, set the applied action to 'deny' otherwise, check if attackData.slowPostAction equals to 'A', set the applied action to 'abort' otherwise, set the applied action to 'alert'
    $eventClassId
    If $appliedAction in ('alert', 'monitor') then 'detect' otherwise, 'mitigate'
    $name
    If $eventClassId = 'detect' then 'Activity detected' otherwise 'Activity mitigated'
    $severity
    If $eventClassId = 'detect' then 5 otherwise 10
    $requestURL

    If $httpMessage.tls is not empty then request = 'https://' + $httpMessage.host otherwise request = 'http://' + $httpMessage.host

    If $httpMessage.path is not empty then request = $request + $attackData.path if $httpMessage.query is not empty then request = $request + '?" + $httpMessage.query

    CEF fields

    CEF Field JSON Field/Value Description
    Device Vendor Akamai  
    Device Version 1.0  
    Device Product akamai_siem  
    Device Event Class ID $eventClassId calculated field
    Name $name calculated field
    Severity $severity calculated field

    CEF Extension fields

    CEF Extension JSON Field/Value Description
    act $appliedAction calculated field
    app $httpMessage.protocol  
    c6a2 $ipv6src IP v6 address of the source. Only populated if $attackData.clientIP is in IP v6 format.
    c6a2Label Source IP6 Address  
    cs1 $attackData.rules Rule IDs of rules that triggered for this request
    cs1Label Rules  
    cs2 $attackData.ruleMessages Messages of rules that triggered for this request
    cs2Label Rule Messages  
    cs3 $attackData.ruleData User data of rules that triggered for this request
    cs3Label Rule Data  
    cs4 $attackData.ruleSelectors Selectors of rules that triggered for this request
    cs4Label Rule Selectors  
    cs5 $attackData.clientReputation Client IP scores for Client Reputation
    cs5Label Client Reputation  
    cs6 $attackData.apiId API ID for API Protection
    cs6Label API ID  
    devicePayloadId $httpMessage.requestId A globally unique ID created to identify this specific message
    dhost $httpMessage.host The value of the host header of the incoming client request
    dpt $httpMessage.port The port number used by the incoming request. Should be equal to the value of AK_IN_PORT
    flexString1 $attackData.configId The ID of the Security Configuration applied to this request
    flexString1Label Security Config Id  
    flexString2 $attackData.policyId This is the id of the Firewall Policy applied to this request
    flexString2Label Firewall Policy Id  
    out $httpMessage.bytes The content bytes served int eh client response
    request $requestURL calculated field
    requestMethod $httpMessage.method The HTTP method of the incoming request
    src $attackData.clientIP The IP address of the client that connects to make the request
    start $httpMessage.start This is the time, in epoch format, to millisecond precision, when the Edge Server initiated the connection for the message exchange being monitored
    AkamaiSiemSlowPostAction $attackData.slowPostAction If a Slow POST attack is detected, this shows the action taken: either W for Warn or A for deny (abort)
    AkamaiSiemSlowPostRate $attackData.slowPostRate If a Slow POST attack is detected, this shows the recorded rate of the Slow POST attack
    AkamaiSiemRuleVersions $attackData.ruleVersions Versions of rules that triggered for this request, base64-encoded
    AkamaiSiemRuleTags $attackData.ruleTags Tags of rules that triggered for this request, base64-encoded
    AkamaiSiemApiKey $attackData.apiKey API Key for API Protection
    AkamaiSiemTLSVersion $httpMessage.tls TLS version, if applicable
    AkamaiSiemRequestHeaders $httpMessage.requestHeaders All request headers collected
    AkamaiSiemResponseHeaders $httpMessage.responseHeaders All response headers collected
    AkamaiSiemResponseStatus $httpMessage.status The HTTP Response status sent to the client
    AkamaiSiemContinent $geo.continent A 2-letter code for the continent that the IP address maps to
    AkamaiSiemCountry $geo.country An ISO-3166, 2-letter code for the state, province, or region where the IP address maps to
    AkamaiSiemCity $geo.city City that the IP address maps to
    AkamaiSiemRegion $geo.regionCode An ISO-3166, 2-letter code for the state, province, or region where the IP address maps to
    AkamaiSiemASN $geo.asn AS number or numbers that the IP belongs to
    AkamaiSiemCusomData $custom Base64 custom formatted value. Size limit is 2KB

    Retrieve past security events using the CEF connector

    Akamai’s CEF connector offers 2 modes of operation:

    • Offset-based. Under normal circumstances, this is the mode you want. The connector automatically logs security events as they’re collected. The connector operates in offset mode when akamai.data.timebased configuration is set to false .
    • Time-based. If your SIEM connection is disrupted, you may want to go back and replay events. You can retrieve security event data that occurred within the last 12 hours.

    To retrieve missing/past events, switch to a time-based feed:

    1. Open your connector’s configuration file: config/CEFConnector.properties .
    2. Change the akamai.data.timebased configuration to true .
    3. Under akamai.data.timebased.from , enter the start time (in epoch format) for which you want to pull past security events. This can be any time within the 12 hours preceding the present moment.
    4. (optional) If you want to retrieve events that fall only within a set time period, under akamai.data.timebased.to , enter the end time (in epoch format) for the period. If you enter nothing here, the connector pulls events up to the present and continues to log events as they’re collected until the connector is restarted.

    If regular offset event collection occurred within the time window you set, duplicate data may appear in your SIEM reports.

    Don't see the data you expected? When you set akamai.data.timebased.from and akamai.data.timebased.to to retrieve security events for a specific period, the connector makes only 1 call to the API. If the number of events in the specified time window exceeds the value in the Limit field (or the default limit of 10,000) the connector won't retrieve data. As a workaround, decrease the time window to include all events.

    To return the connector to offset mode, change akamai.data.timebased configuration back to false.

    Uninstall CEF connector

    Stop the Akamai CEF Connector service by running bin/AkamaiCEFConnector.sh stop or /etc/init.d/symboliclink stop

    Remove the installation folder by running rm –rf CEFConnector-1.0

    Release notes

    CEF Connector version 1.5 released in August 2017. Changes include:

    • Debug flag has moved out of CEFConnector.properties. To turn on logging, you now make a quick change to the log4j2.xml. To read how, see Logging.
    • All logs now consolidated in the /bin/logs/cefconnector.log file.