Blog

Create a DevOps Pipeline with the Akamai Docker Image

June 1, 2020 · by Lukasz Czerpak ·
Categories:

The Akamai command-line interface (CLI) is a powerful and extensible toolkit that allows you to manage and configure Akamai's platform and various products directly from the command line. For those of you doing DevOps, the CLI simplifies the integration of Akamai into your existing CI/CD automation. It delivers a unified developer experience for anyone who needs to work with or automate Akamai.

Various CLI modules can be developed in different programming languages like Python, Go, or Javascript. This gives a lot of flexibility, but at the same may require additional effort

Management of artifacts on Akamai Platform like properties, firewall rules, cpcode and so on, can significantly be simplified by using Akamai CLIs. It's a powerful and extensible toolkit that allows you to manage and configure Akamai's platform and various products directly from the command line. It can be used during local development as well as in automated CI/CD pipelines.

In this blog post I will show you a few examples how Akamai CLIs can be used efficiently by using Akamai Docker Image.

Local Development and Testing

Akamai properties can be managed in various ways. One of the approaches is to use Akamai Pipeline CLI and decompose the whole configuration into functional fragments that are stored in separate files on the filesystem. This approach is also better if code is stored in VCS like Git.

Listing below illustrates structure of my working directory - folder pipeline is stored in Git and contains files for my property - it has been created with Akamai Pipeline CLI:

For more on how to set up the Akamai Docker Development Environment, please check out our blog.

├── pipeline
│   ├── Jenkinsfile
│   ├── demo
│   │   ├── cache
│   │   ├── dist
│   │   │   └── lczerpak-a2s.stg.webperf.it.papi.json
│   │   ├── environments
│   │   │   ├── lczerpak-a2s.stg.webperf.it
│   │   │   │   ├── envInfo.json
│   │   │   │   ├── hostnames.json
│   │   │   │   └── variables.json
│   │   │   └── variableDefinitions.json
│   │   ├── projectInfo.json
│   │   └── templates
│   │       ├── Assets.json
│   │       └── main.json
│   └── devops-logs.log
└── sandboxes

My sample website was refactored and images changed location. This needs to be reflected in Akamai configuration otherwise images won't be cached.

Fix for image caching is already in my workspace.

First, I added a separate file with a rule for caching images. And then included this file in the main JSON file. I also added a sample image URL to the test suite to ensure that it will be properly tested by the pipeline in Jenkins.

❯ git diff

diff --git a/Jenkinsfile b/Jenkinsfile

index 622bcb1..fccbe27 100644

--- a/Jenkinsfile

+++ b/Jenkinsfile

@@ -92,7 +92,8 @@ EOF

      [

        'http://gcs.stg.webperf.it/assets/css/main.css',

-        'http://gcs.stg.webperf.it/assets/js/main.js'

+        'http://gcs.stg.webperf.it/assets/js/main.js',

+        'http://gcs.stg.webperf.it/images/pic02.jpg'

      ].each { url ->

        def headers = sh(

          returnStdout: true,

diff --git a/demo/templates/Images.json b/demo/templates/Images.json

new file mode 100644

index 0000000..59acf36

--- /dev/null

+++ b/demo/templates/Images.json

@@ -0,0 +1,27 @@

+{

+    "name": "Images",

+    "children": [],

+    "behaviors": [

+        {

+            "name": "caching",

+            "options": {

+                "behavior": "MAX_AGE",

+                "mustRevalidate": false,

+                "ttl": "7d"

+            }

+        }

+    ],

+    "criteria": [

+        {

+            "name": "path",

+            "options": {

+                "matchOperator": "MATCHES_ONE_OF",

+                "values": [

+                    "/images/*"

+                ],

+                "matchCaseSensitive": false

+            }

+        }

+    ],

+    "criteriaMustSatisfy": "all"

+}

diff --git a/demo/templates/main.json b/demo/templates/main.json

index 656902b..4bc16ae 100644

--- a/demo/templates/main.json

+++ b/demo/templates/main.json

@@ -2,7 +2,8 @@

    "rules": {

        "name": "default",

        "children": [

-            "#include:Assets.json"

+            "#include:Assets.json",

+            "#include:Images.json"

        ],

        "behaviors": [

            {

Before this change goes to the repository, it needs to be tested locally. We will use Akamai Sandbox in Akamai Docker Image for this task.

Environment setup

Since multiple sandbox commands will be used, we will create long running docker container with all paths and config files mapped:

❯ docker run --name akabox -it --rm -p 9550:9550 -v $HOME/.edgerc:/root/.edgerc:ro -v $(pwd)/sandboxes:/sandboxes -v $(pwd)/pipeline:/workspace akamai/akamai-docker cat

The above container will be up and running until cat process is killed (by pressing ctrl+d for instance).

Now, we need to create a new sandbox for the property:

❯ docker exec -it akabox akamai sandbox create --name webinar_demo --property lczerpak-a2s.stg.webperf.it

building origin list

Detected the following origins: kr.czerpak.eu

? Do you want the Sandbox Client to proxy the origins in your dev environment to the destination defined in the Akamai config? Enter **y** and the CLI will automatically update your configuration file. If

you want to route sandbox traffic to different development origins, enter **n** to customize the origin mappings. Yes

registering sandbox in local datastore

sandbox_id: 643c3722-cb26-48d4-9ed1-c408e138157e webinar_demo is now active

Successfully created sandbox_id 643c3722-cb26-48d4-9ed1-c408e138157e Generated sandbox client configuration at /cli/.akamai-cli/cache/sandbox-cli/sandboxes/webinar_demo/config.json Edit this file to specify the port and host for your dev environment.

Please note that all commands are executed in the same container that we've created first. This saves me some effort on added .edgerc file and other folder mappings to each command. It's just more convenient

Once the sandbox has been created, we can start it:

❯ docker exec -it akabox akamai sandbox start

launching client with command: "/usr/lib/jvm/default-jvm/bin/java" -Dlogging.path="/cli/.akamai-cli/cache/sandbox-cli/sandboxes/webinar_demo/logs" -Dlogging.config="/cli/.akamai-cli/cache/sandbox-cli/sandbox-client-1.2.0-RELEASE/conf/logback.xml" -jar "/cli/.akamai-cli/cache/sandbox-cli/sandbox-client-1.2.0-RELEASE/lib/sandbox-client-1.2.0-RELEASE.jar" --config="/cli/.akamai-cli/cache/sandbox-cli/sandboxes/webinar_demo/config.json"​

 .   ____          _            __ _ _
/\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/  ___)| |_)| | | | | || (_| |  ) ) ) )
 '  |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot ::        (v1.5.1.RELEASE)

2020-05-20 12:52:45 [main] INFO  c.a.devpops.connector.ConnectorMain - Starting ConnectorMain v1.2.0-RELEASE on fc70c4c87cd7 with PID 95 (/cli/.akamai-cli/cache/sandbox-cli/sandbox-client-1.2.0-RELEASE/lib/sandbox-client-1.2.0-RELEASE.jar started by root in /root)

2020-05-20 12:52:45 [main] INFO  c.a.devpops.connector.ConnectorMain - No active profile set, falling back to default profiles: default

2020-05-20 12:52:45 [main] INFO  o.s.c.a.AnnotationConfigApplicationContext - Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@39a054a5: startup date [Wed May 20 12:52:45 GMT 2020]; root of context hierarchy

2020-05-20 12:52:47 [main] INFO  c.a.d.c.service.OriginTargetService - parsing origin mappings

2020-05-20 12:52:48 [main] INFO  o.s.s.c.ThreadPoolTaskExecutor - Initializing ExecutorService  'taskExecutor'

2020-05-20 12:52:48 [main] INFO  o.s.j.e.a.AnnotationMBeanExporter - Registering beans for JMX exposure on startup

2020-05-20 12:52:48 [main] INFO  c.a.devpops.connector.ConnectorMain - Setting simple memory leak detector

2020-05-20 12:52:48 [main] INFO  c.a.devpops.connector.ConnectorMain - started internal channel - internalOriginRequestDispatcher

2020-05-20 12:52:48 [main] INFO  c.a.devpops.connector.ConnectorMain - starting HTTP 1.1 server

2020-05-20 12:52:48 [main] INFO  c.a.devpops.connector.ConnectorMain - started primary http channel

2020-05-20 12:52:48 [main] INFO  c.a.devpops.connector.ConnectorMain - Started ConnectorMain in 4.566 seconds (JVM running for 6.307)

2020-05-20 12:52:48 [Timer-0] INFO  c.a.devpops.connector.ConnectorMain - Successfully launched Akamai Sandbox Client

2020-05-20 12:52:48 [Timer-0] INFO  c.a.devpops.connector.ConnectorMain - Sandbox Client running on port: 9550

and check if it's working - also need to know its identifier for further invocations:

❯ docker exec -it akabox akamai sandbox list

Local sandboxes:

current  name          sandbox_id

-------  ------------  ------------------------------------

YES      webinar_demo  643c3722-cb26-48d4-9ed1-c408e138157e

Testing the change

At this point, the sandbox is using settings from the property with images not being cached. We can easily validate it by comparing X-Check-Cacheable response header for Javascript assets vs. Image.

❯ curl -ks -IXGET -H "Pragma: akamai-x-check-cacheable" --connect-to ::localhost:9550 http://gcs.stg.webperf.it/assets/js/main.js 2>&1 | grep -i x-check-cacheable

X-Check-Cacheable: YES

❯ curl -ks -IXGET -H "Pragma: akamai-x-check-cacheable" --connect-to ::localhost:9550 http://gcs.stg.webperf.it/images/pic02.jpg 2>&1 | grep -i x-check-cacheable

X-Check-Cacheable: NO

It clearly shows the problem - it's set to YES for the Javascript and set to NO for Image.

Let's build fixed config from local files:

❯ docker exec -it -w /workspace akabox akamai pipeline merge -n -p demo lczerpak-a2s.stg.webperf.it
╒══════════════════════╤═════════════════════════════════════════════════════════════╕
│"Action"              │"Result"                                                     │
╞══════════════════════╪═════════════════════════════════════════════════════════════╡
│"changes detected"    │"yes"                                                        │
├──────────────────────┼─────────────────────────────────────────────────────────────┤
│"rule tree stored in" │"/workspace/demo/dist/lczerpak-a2s.stg.webperf.it.papi.json" │
├──────────────────────┼─────────────────────────────────────────────────────────────┤
│"hash"                │"7cad176b4232e3d2aeb64 . . . . .1b4164bf2bf72ec569ef6e5383c1"│
├──────────────────────┼─────────────────────────────────────────────────────────────┤
│"validation performed"│"no"                                                         │
├──────────────────────┼─────────────────────────────────────────────────────────────┤
│"validation warnings" │"no"                                                         │
├──────────────────────┼─────────────────────────────────────────────────────────────┤
│"validation errors"   │"no"                                                         │
├──────────────────────┼─────────────────────────────────────────────────────────────┤
│"hostname warnings"   │"no"                                                         │
├──────────────────────┼─────────────────────────────────────────────────────────────┤
│"hostname errors"     │"no"                                                         │
└──────────────────────┴─────────────────────────────────────────────────────────────┘

And deploy to the sandbox instance:

❯ docker exec -it akabox akamai sandbox update -r /workspace/demo/dist/lczerpak-a2s.stg.webperf.it.papi.json 643c3722-cb26-48d4-9ed1-c408e138157e

Successfully updated sandbox_id: 643c3722-cb26-48d4-9ed1-c408e138157e

If we test the same image URL, we see that the caching is enabled which proves that the fix works as expected:

❯ curl -ks -IXGET -H "Pragma: akamai-x-check-cacheable" --connect-to ::localhost:9550 http://gcs.stg.webperf.it/assets/js/main.js 2>&1 | grep -i x-check-cacheable

X-Check-Cacheable: YES

❯ curl -ks -IXGET -H "Pragma: akamai-x-check-cacheable" --connect-to ::localhost:9550 http://gcs.stg.webperf.it/images/pic02.jpg 2>&1 | grep -i x-check-cacheable

X-Check-Cacheable: YES

We have confidence that the change is working fine and can go to the repository:

❯ git commit -m "fixed caching for images"

[master 9c83b42] fixed caching for images

3 files changed, 31 insertions(+), 2 deletions(-)

create mode 100644 demo/templates/Images.json

❯ git push

Enumerating objects: 12, done.

Counting objects: 100% (12/12), done.

Delta compression using up to 8 threads

Compressing objects: 100% (7/7), done.

Writing objects: 100% (7/7), 825 bytes | 825.00 KiB/s, done.

Total 7 (delta 3), reused 0 (delta 0), pack-reused 0

To ssh://localhost:2222/lczerpak/demo.git

  e199aec..9c83b42  master -> master

CI/CD pipeline in Jenkins

The change we've just pushed to the repository is automatically picked up by Jenkins:

Jenkins

It executes similar tasks we've done in terminal like building the config, pushing to the network and unit testing, but it's fully automated:

Demo

Let's have a look how it was setup in the Jenkinsfile:

node {

 properties([

   pipelineTriggers([

     [$class: 'GenericTrigger',

       genericVariables: [

         [key: 'ref', value: '$.ref'],

         [key: 'before', value: '$.before']

       ],

       genericRequestVariables: [

         [key: 'requestWithNumber', regexpFilter: '[^0-9]'],

         [key: 'requestWithString', regexpFilter: '']

       ],

       genericHeaderVariables: [

         [key: 'headerWithNumber', regexpFilter: '[^0-9]'],

         [key: 'headerWithString', regexpFilter: '']

       ],

       causeString: 'Triggered on $ref',

       token: 'lczerpak-jenkins-webinar',

       printContributedVariables: true,

       printPostContent: true,

       silentResponse: false,

       regexpFilterText: '$ref',

       regexpFilterExpression: 'refs/heads/' + BRANCH_NAME

     ]

   ])

 ])

 def CPCODE = 469271

 def PIPELINE_NAME = 'demo'

 def PROPERTY_NAME = 'lczerpak-a2s.stg.webperf.it'

 String versionNotes

 stage('Checkout') {

   step([$class: 'WsCleanup'])

   checkout scm

 }

 docker.image("akamai/akamai-docker").inside() {

   stage('Prepare') {

     withCredentials([

       string(credentialsId: 'gcs-host', variable: 'AK_HOST'),

       string(credentialsId: 'gcs-access-token', variable: 'AK_ACCESS_TOKEN'),

       string(credentialsId: 'gcs-client-token', variable: 'AK_CLIENT_TOKEN'),

       string(credentialsId: 'gcs-client-secret', variable: 'AK_CLIENT_SECRET'),

       string(credentialsId: 'gcs-ccu-host', variable: 'AK_CCU_HOST'),

       string(credentialsId: 'gcs-ccu-access-token', variable: 'AK_CCU_ACCESS_TOKEN'),

       string(credentialsId: 'gcs-ccu-client-token', variable: 'AK_CCU_CLIENT_TOKEN'),

       string(credentialsId: 'gcs-ccu-client-secret', variable: 'AK_CCU_CLIENT_SECRET')

     ]) {

       sh("""cat <<EOF | tee /root/.edgerc

[default]

client_secret=${AK_CLIENT_SECRET}

host=${AK_HOST}

access_token=${AK_ACCESS_TOKEN}

client_token=${AK_CLIENT_TOKEN}

[ccu]

client_secret=${AK_CCU_CLIENT_SECRET}

host=${AK_CCU_HOST}

access_token=${AK_CCU_ACCESS_TOKEN}

client_token=${AK_CCU_CLIENT_TOKEN}

EOF

       """)

     }

     versionNotes = sh(returnStdout: true, script: 'git --no-pager log --pretty="%h: %s" | head -1 | tr -d "\n"')

   }

   stage('Build') {

     sh("""akamai pipeline merge -n -p ${PIPELINE_NAME} ${PROPERTY_NAME}""")

     sh("""akamai property update ${PROPERTY_NAME} --file ${PIPELINE_NAME}/dist/${PROPERTY_NAME}.papi.json --notes "${versionNotes}" --section default""")

     archiveArtifacts artifacts: "${PIPELINE_NAME}/dist/${PROPERTY_NAME}.papi.json", fingerprint: true

   }

   stage('Deploy to Staging') {

     sh("""akamai property activate ${PROPERTY_NAME} --email lczerpak@akamai.com --notes "${versionNotes}" --network staging --section default""")

   }

   stage('Purge Staging') {

     sh("""akamai purge --section ccu delete --staging --cpcode ${CPCODE}""")

     echo 'Waiting 10 seconds for purge to complete'

     sleep(time:10, unit:"SECONDS")

   }

   stage('Testing Staging') {

     def error = false

     [

       'http://gcs.stg.webperf.it/assets/css/main.css',

       'http://gcs.stg.webperf.it/assets/js/main.js',

       'http://gcs.stg.webperf.it/images/pic02.jpg'

     ].each { url ->

       def headers = sh(

         returnStdout: true,

         script: """curl -ks -IXGET -H "Pragma: akamai-x-check-cacheable" --connect-to ::webperf.it.edgesuite-staging.net: ${url}""").trim()

       // Check if STAGING

       if (!headers.contains("X-Akamai-Staging: EdgeSuite")) {

           echo "ERROR: Not going through STAGING"

           error = true

       }

       // Check if CACHEABLE

       if (!headers.contains("X-Check-Cacheable: YES")) {

           echo "ERROR: '${url}' not cachable"

           error = true

       }        

     }

     if (error) {

       throw new Exception("Testing on STAGING failed!")

     }

   }

 }

}

Almost all stages and steps defined in the pipeline are executed inside Akamai Docker Container. It gives access to all functionalities offered by Akamai CLIs. This sample implementation uses Akamai Pipeline CLI to build the config, Akamai Property CLI to deploy it to Akamai Network, Akamai Purge CLI to remove content from cache before testing.

Summary

Akamai Docker Image can extremely simplify not only local development but also automation in CI/CD systems. This unified way of using Akamai DevOps tools doesn't involve any additional dependency management since all is encapsulated in the image. That allows for almost zero-cost adoption of Akamai DevOps tools in your organization.

You Might Also Like:

About the author

Lukasz

Lukasz Czerpak is a Senior Enterprise Architect at Akamai based in Krakow, Poland with 10+ years of experience designing, developing and managing software projects, ISP infrastructure, and cloud services. Lukasz works with Akamai customers on some of our most advanced and complex integrations, he also provides consulting and DevOps training, and is a regular speaker at Technology Days.