Blog

Automatically Clone Your Config with PAPI

August 2, 2017 by John Councilman

There are times when you want to automatically clone and edit a configuration, rather than doing it manually through Luna. By using the Property Manager API (PAPI), you can do exactly that. This is particularly beneficial in the case where you have a "Master Config" with all the required optimizations that you want to use as a template for creating many similar sites or streaming configurations.

This blog post will show you how to clone your config with PAPI for the above use cases.

For this blog post, I've chosen to use Python, as many of my clients are using this for automation in place of Perl or Java. The Akamai developer portal has libraries for several different languages, so if you want to use something else, see Open Source API Clients.

In the rest of this post I'll run through an example workflow utilizing Python, and explain each step.

 

Prerequisites

An admin with Luna access must obtain the authentication information needed to interact with an account using Akamai APIs. Specifically, you’ll need the following information:

  • Base URL
  • Client TokenC
  • Client Secret
  • Access Token

You'll see these values in the Luna interface. Details on obtaining this information may be found here: https://developer.akamai.com/introduction/Prov_Creds.html

 

Obtain Libraries and Initialize

 

Begin by initializing Python as you would any other script and importing libraries:

 

#!/usr/bin/python

Next, obtain the Akamai library from https://github.com/akamai-open/AkamaiOPEN-edgegrid-python.

Now Initialize Requests (HTTP) object and Akamai Auth:

import requests from akamai.edgegrid import EdgeGridAuth

Next import URLJoin (available via pip) to easily create full URLs from the Akamai base URL.

from urlparse import urljoin

Next, you utilize the JSON module. This is mainly used for generating JSON requests outside of HTTP Requests. HTTP Requests contains its own built-in JSON Parser. Available via Pip as well.

import json

Because we must parse some strings returned by the API which do not utilize JSON, I’ve decided to utilize the regular expression module (available via Pip) to easily parse strings returned.

import re

This script is interactive. We would like for a user to be able to utilize command-line arguments to interact with the script instead of hard-coding everything. The argparse module (Available via Pip) allow this to be easily implemented.

import argparse

 

Initialize Authentication and Variables

Use the authentication credentials from the Prerequisites section to enter the Base URL.

baseurl = 'https://akab-REDACTED.luna.akamaiapis.net/' s = requests.Session()

Now enter the client secret information is entered here. For this example, we are statically assigning it within the code, but you could keep it in an external file with some sort of encryption algorithm to secure the information.

s.auth = EdgeGridAuth( client_token='REDACTED', client_secret='REDACTED', access_token='REDACTED' )

 

Adding Interactive Command-line Arguments

The following bit of code will contain all interactive command line arguments to be utilized later within the script. Note that you could hard code all the values and not make it interactive. In this case the argparse module would not be needed.

Notice that we also implement a “list info” option to display all current configurations. This will be explained later.

We’ve added a couple of options below to allow for overwrite of specific property manager variables which will change for each newly cloned configuration; specifically Segment Duration, Live/On-Demand, and Authentication Information.

parser = argparse.ArgumentParser(description='Automatically clone Akamai configurations.')
parser.add_argument('--listinfo', help='Prints all Group and Property info if set.',action='store_true',default='False')
parser.add_argument('--property', help='Property ID to clone from',required=False)
parser.add_argument('--newgroup', help='New groupID to creat property in',required=False)
parser.add_argument('--suffix', help='Akamai suffix to use (Example: akamaihd.net)',required=False)
parser.add_argument('--email', help='Email address to receive activation notifications to. use --email multiple time for multiple addresses',required=False,action='append')
parser.add_argument('--secure', help='true or false, whether to use HTTPS',required=False)
parser.add_argument('--name', help='Property name to use for new property (Example: test.nhl.com)',required=False)
parser.add_argument('--live', help='0 for Live; 1 for OnDemand (HTTP Downloads Only)',required=False)
parser.add_argument('--segdur', help='Number of seconds for segments (Example: 6) (HTTP Downloads Only)',required=False)
parser.add_argument('--useauth', help='0 to diable authentication; 1 to enable tokenization (HTTP Downloads Only)',required=False)
parser.add_argument('--origin', help='Hostname of Origin server',required=False)
parser.add_argument('--nscp', help='CPCode for Netstorage when using Netstorage as Origin server',required=False)
parser.add_argument('--cpcode', help='Optional: Override CP Code with pre-created one',required=False,default='auto')

 

Typically, we would only clone configurations residing within a single contract. It’s safe to statically declare a contract ID within the code. Your contract ID may be contained from your account contact or within Luna.

contractId = "ctr_3-XXXXX"

Initialize the command line argument parser:

args = parser.parse_args()

 

List Group and Property Information (Alternate Flow)

We’ve created a specific use case within the script lying outside of the configuration flow utilized to list all property and group IDs within a contract. This is useful for determining your environment, source Group IDs, destination Group IDs, and Source Property IDs to populate the script with for actual configuration.

The following will pull all groups within a contract, then loop through available properties within each respective group. Notice that the built-in JSON parser with “requests” is used.

if args.listinfo is True:
print "Obtaining groups and properties available...\n"
endurl = "/papi/v0/groups/"
result = s.get(urljoin(baseurl, endurl))
groups = result.json()['groups']['items']
for items in groups:
testGroup = items['groupId']
testName = items['groupName']
endurl = "/papi/v0/properties/?contractId=" + contractId +"&groupId=" + testGroup
result = s.get(urljoin(baseurl, endurl))
if result.status_code != 403:
print "----Group----"
print "GroupID : " + testGroup
print "GroupName : " + testName + "\n"
properties = result.json()['properties']['items']
for items2 in properties:
print "\tProperty ID : " + items2['propertyId'] + " (" + items2['propertyName'] + ")"
print "\n\n\n"
exit(0)

The output will look similar to the following:

Obtaining groups and properties available...

----Group---- G
roupID : grp_12345
GroupName : WebSites

Property ID : prp_12345 (www.website.com)
Property ID : prp_54321 (streaming.website.com)
Property ID : prp_13579 (photos.website.com)

 

Continuation of Variable Assignment if Cloning Property

Below, we assign variables from the command line arguments or exit if required arguments are not given.

if not args.name or not args.email or not args.newgroup or not args.property or not args.secure or not args.origin: parser.print_help() print "\n\n" print "If you are unaware of which properties and groups exist in order to" print "populate this script, please run with --listinfo flag" exit(0)

if "download.akamai.com" in str(args.origin) and not args.nscp: print "Netstorage CPCode required when using Netstorage origin\n" exit(0)

########## Begin initialization variables: ################ ### Save CP Code from CLI entry cpId = args.cpcode

### Netstorage CP Code nsCP = args.nscp

### Origin Server originServer = args.origin

### The is the group ID containing Master Configs - Static groupId = "grp_111111"

### This is the group ID we are creating the new configuration in. Use "List Groups below to find entire listing newGroupId = args.newgroup

### Property ID of Master Config used as "Master". "List Properties" below shows all properties in a particular group. propertyId = args.property

### New Property name to create. This would be input into the script, using command line variables, CGI input, etc. newProperty = args.name

### Depends on source domain. edgesuite.net, akamaized.net, etc are all OK domainSuffix = args.suffix

### True or False. Whether HTTPS is supported on this config. secure = args.secure

### IPV6_COMPLIANCE or IPV4 are your choices here ### Hardcoded IPV6 Support Static ipVersion = "IPV6_COMPLIANCE"

### Email to receive activation notificaions userEmail = args.email

### Set Property Manager variables here for HTTP Downloads only ### Is this a Live or On-Demand stream (0 = OnDemand; 1 = Live) isLive = args.live

### .ts segment duration in seconds segDuration = args.segdur

### If authenticated stream, 1, if clear, 0 useAuth = args.useauth

 

Initialize Authentication and Variables

The first thing we must do when cloning a configuration is to obtain the latest version of the Source configuration, referred to as the Master Config.

Obtain Master Config Version, ProductID, and eTag

Two examples are given below, one obtains the absolute latest version; the other obtains the latest Production version. Utilize one or the other - choosing the one best for your use case.

Latest PRODUCTION version:

#endurl = "/papi/v0/properties/" + propertyId + "/versions/latest?contractId=" + contractId + "&groupId=" + groupId + "&activatedOn=PRODUCTION"

Absolute latest version:

endurl = "/papi/v0/properties/" + propertyId + "/versions/latest?contractId=" + contractId + "&groupId=" + groupId

In either case, use:

result = s.get(urljoin(baseurl, endurl)) Debug info: May be removed or commented: print "Master config version info : \n" + result.text

Expect a 200 status code for success, otherwise, error out:

if result.status_code != 200:
print "Error obtaining version and Product\n"
exit(0)

Utilize the built-in JSON parser to extract the needed variables:

version = str(result.json()['versions']['items'][0]['propertyVersion']) productId = result.json()['versions']['items'][0]['productId'] eTag = result.json()['versions']['items'][0]['etag']

Finally, here's an example to delete a property prior to cloning. This is not normally used, but documented for example only:

### Not needed, example only ### Delete Property -- Only used when testing over and over print "Delete old property" endurl = "/papi/v0/properties/prp_363012?contractId=ctr_3-Z54RW5&groupId=grp_95552" result = s.delete(urljoin(baseurl,endurl)) print "Result " + result.text

 

Create a New Property Based on Master Config

First off, we should create a new cloned configuration based on the original Master Configuration. This configuration will require modification once created.

### Create new property based on old config string1 = productId string2 = newProperty string3 = propertyId string4 = version string5 = eTag

Notice how we are creating a JSON structure as a variable. We will do this several more times.

send_data = """ { "productId": "%s", "propertyName": "%s", "cloneFrom": { "propertyId": "%s", "version": %s, "cloneFromVersionEtag": "%s" } } """ % (string1, string2, string3, string4, string5) ### Uncomment to see data sent print "Sending new property command :\n" + send_data

Below, notice newGroupId in the request URL. This is where we are creating new property (outside of Master Config group and in another one).

endurl = "/papi/v0/properties/?contractId=" + contractId + "&groupId=" + newGroupId headers = {'Content-Type': 'application/json','Pragma': 'edgegrid-fingerprints-on'} print "Sending to URL " + urljoin(baseurl,endurl) result = s.post(urljoin(baseurl, endurl),data=send_data,headers=headers) Status code and result data from operation: print "Status Code is " + str(result.status_code) print "Result is :\n" + result.text

Note that the status code from the Clone command is 201; not 200.

if result.status_code != 201: print "Error creating new Property\nDetails:\n" print result.text exit(0) propertyLink = result.json()['propertyLink']

You need to use a regular expression match to pull out only the property ID from propertyLink output:

nPropId = re.findall(r'(prp_\d+)', propertyLink) nPropId = nPropId[0]

print "New property Created with ID: " + nPropId

 

Once the property has been created, we must create a new hostname to attach to the property.

string1 = productId string2 = newProperty string3 = domainSuffix string4 = secure string5 = ipVersion

send_data = """ { "productId": "%s", "domainPrefix": "%s", "domainSuffix": "%s", "secure": "%s", "ipVersionBehavior": "%s" } """ % (string1, string2, string3, string4, string5) print "Sending Hostname command :\n" + send_data

endurl = "/papi/v0/edgehostnames/?contractId=" + contractId +"&groupId=" + newGroupId headers = {'Content-Type': 'application/json','Pragma': 'edgegrid-fingerprints-on'} result = s.post(urljoin(baseurl, endurl),data=send_data,headers=headers) print "Status Code is " + str(result.status_code) print "Result is " + result.text

Again, the status code of a successful operation will be 201 after the Create Hostname command. If we receive something other than a 201, we might have created this hostname before, so let’s try to re-use it first before erroring.

if result.status_code != 201: print "Hostname already likely exists. Find existing link" endurl = "/papi/v0/edgehostnames/?contractId=" + contractId + "&groupId=" + newGroupId result = s.get(urljoin(baseurl, endurl)) print "Checking for hostname: " + result.text groups = result.json()['edgeHostnames']['items'] for items in groups: if items['domainPrefix'] == newProperty: ehn = str(items['edgeHostnameId']) print "EHN is: " + ehn

In the case of 201, we have successfully created a new hostname:

elif result.status_code == 201: print " Debug : " + result.text edgeHostnameLink = result.json()['edgeHostnameLink'] ehn = re.findall(r'(ehn_\d+)', edgeHostnameLink) ehn = ehn[0]

match = re.match(r'ehn',ehn,re.I) if not match: print "Error: Creating Edge Hostname" exit(0)

print "Edge Hostname is " +ehn

Once a new configuration and hostname have been created, we should pull the newest version of the newly cloned configuration. This is similar transaction as the one to pull the version of the Master configuration. Notice that we do not ask for the Production version since we've never activated this one. We also must pull the eTag from this particular configuration for later use.

### Get new version of NEW Config endurl = "/papi/v0/properties/" + nPropId + "/versions/latest?contractId=" + contractId + "&groupId=" + newGroupId print baseurl + endurl result = s.get(urljoin(baseurl, endurl)) print "Get new version results :\n" + result.text newVersion = str(result.json()['versions']['items'][0]['propertyVersion']) newEtag = str(result.json()['versions']['items'][0]['etag'])

print "New Version is " + newVersion print "New Etag is " + newEtag

Using the version number obtained before along with the hostname link, we will attach the newly-created Hostname to the newly-created property. Note that this particular request results in a 200 status code on success.

### Attach Hostname to Property

string1 = ehn string2 = newProperty

send_data = """ [ { "cnameType": "EDGE_HOSTNAME", "edgeHostnameId": "%s", "cnameFrom": "%s" } ] """ % (string1, string2) print "Sending data to attach hostname :\n" + send_data

endurl = "/papi/v0/properties/" + nPropId +"/versions/" + newVersion + "/hostnames?contractId=" + contractId + "&groupId=" + newGroupId headers = {'Content-Type': 'application/json','Pragma': 'edgegrid-fingerprints-on'} result = s.put(urljoin(baseurl, endurl),data=send_data,headers=headers)

print "Results :\n" + result.text

if result.status_code != 200: print "Error attaching hostname to property\n" print result.text exit(0)

### Get product ID/Name from new property

endurl = "/papi/v0/properties/" + nPropId + "/versions/" + newVersion + "/rules?contractId=" + contractId +"&groupId=" + newGroupId result = s.get(urljoin(baseurl, endurl)) print "Get Product ID Results : \n" + result.text

# Save full JSON for later binary = result.content output = json.loads(binary)

groups = result.json()['rules']['behaviors'] for items in groups: if items['name'] == "cpCode": cpProduct = "prd_" + str(items['options']['value']['products'][0]) print "cpProduct is " + cpProduct

if cpId == "auto": ### Create new CP Code string1 = newProperty string2 = cpProduct

send_data = """ { "cpcodeName": "%s", "productId": "%s" } """ % (string1, string2)

endurl = "/papi/v0/cpcodes/?contractId=" + contractId + "&groupId=" + newGroupId headers = {'Content-Type': 'application/json','Pragma': 'edgegrid-fingerprints-on'} result = s.post(urljoin(baseurl, endurl),data=send_data,headers=headers)

if result.status_code != 201: print "Error creating new CP Code\n" print result.text exit(0)

cpcodeLink = result.json()['cpcodeLink'] print "CP Code Results :\n" + result.text ### Must use regular expression match to pull out only CP ID from cpcodeLink output cpId = re.findall(r'cpc_(\d+)', cpcodeLink)

cpId = cpId[0] print "New CPCode Created with ID: " + cpId print "CP Code being used : " + cpId

 

Edit Cloned Configuration Prior to Publication

Now that you have your cloned configuration, have attached a new hostname to it, and have generated a new CP Code, you must customize the configuration prior to activation on the network.

Notice that we're modifying a previously obtained JSON structure within memory.

for stuff in output['rules']['variables']: if stuff['name'] == "PMUSER_SEGMENT_DURATION": stuff['value'] = str(segDuration)

if stuff['name'] == "PMUSER_IS_LIVE": stuff['value'] = str(isLive)

if stuff['name'] == "PMUSER_USE_AUTH": stuff['value'] = str(useAuth)

for stuff in output['rules']['behaviors']: if stuff['name'] == "cpCode": cpProduct = "prd_" + str(stuff['options']['value']['products'][0])

for stuff in output['rules']['behaviors']: if stuff['name'] == "cpCode": stuff['options']['value']['id'] = int(cpId) stuff['options']['value']['name'] = newProperty if stuff['name'] == "origin": stuff['options']['hostname'] = newProperty

send_data = json.dumps(output)

print "JSON to send for new property : \n" + send_data

### Publish new config changes endurl = "/papi/v0/properties/" + nPropId +"/versions/" + newVersion + "/rules/?contractId=" + contractId + "&groupId=" + newGroupId headers = {'Content-Type': 'application/json','Pragma': 'edgegrid-fingerprints-on'} result = s.put(urljoin(baseurl, endurl),data=send_data,headers=headers)

print "Result of properly publish :\n" + result.text if result.status_code != 200: print "Error publishing property\n" print result.text exit(0)

 

Activate the Property on the Akamai Network

You'll want to notify people about the new property, use the following code to generate an email list for single or multiple email addresses.

if len(userEmail) >1: chunks = [] for items in userEmail: string = "\"" + items + "\",\n" chunks.append(string)

result = ''.join(chunks) result = result[:-2] userEmail = result

### Use single email in quotes if only one given elif len(userEmail) ==0: userEmail = "\"" + userEmail + "\""

I broke up the code here, because after the email structure the actual activation begins.

string1 = newVersion ### Change to PROCTION to activate in Production. I wouldn't suggest doing this over the API string2 = "STAGING" string3 = "First activation" string4 = userEmail

send_data = """ { "propertyVersion": %s, "network": "%s", "note": "%s", "notifyEmails": [ %s ] } """ % (string1, string2, string3, string4) print "sending data or property activation :\n" + send_data ### Send first activation request to receive warnings endurl = "/papi/v0/properties/" + nPropId + "/activations/?contractId=" + contractId +"&groupId=" + newGroupId headers = {'Content-Type': 'application/json','Pragma': 'edgegrid-fingerprints-on'} result = s.post(urljoin(baseurl, endurl),data=send_data,headers=headers)

### First config activation will result in warning, Grab those to acknowledge print "Results of first activation : \n" + result.text warnings = result.json()['warnings'] chunks = [] for items in warnings: print items['messageId'] string = "\"" + items['messageId'] + "\",\n" chunks.append(string)

result = ''.join(chunks) ### Drop newline and last comma result = result[:-2]

### Activate config in Staging with acklowledged warnings string1 = newVersion string2 = "STAGING" string3 = "First activation" string4 = userEmail string5 = result

send_data = """ { "propertyVersion": %s, "network": "%s", "note": "%s", "notifyEmails": [ %s ], "acknowledgeWarnings": [ %s ] } """ % (string1, string2, string3, string4, string5)

print "Data being sent for second activation request :\n" + send_data ### Send request again with acknowledged warnings endurl = "/papi/v0/properties/" + nPropId + "/activations/?contractId=" + contractId +"&groupId=" + newGroupId headers = {'Content-Type': 'application/json','Pragma': 'edgegrid-fingerprints-on'} result = s.post(urljoin(baseurl, endurl),data=send_data,headers=headers) print "result of second activation :\n" + result.text

 

Categories: