I started with a question: Is there a way to toggle between two dramatically different site capabilities in less than five seconds?
Here’s how I found the answer.
I recently wrote a white paper in which this issue of toggle times appeared. The topic was using your CDN to control dark releases. The challenge implementing the solutions described in the white paper was how fast the desired feature could be enabled. One configuration change, for example, could take up to 15 minutes to propagate, and that is a long time—too long for some organizations.
While I was thinking about this issue of unacceptably slow configuration changes, I came across work done by my Akamai colleague Thomas Gleason where he used DNS to control configuration behavior. That gave me an idea: Akamai servers have the capability to expose in a variable the CNAMEs behind a domain, and that information could be used to perform matches and enable/disable features, thus potentially speeding up configuration changes.
Let’s see how this works using the example of a simplified search. As explained in an earlier blog post on this site about best practices for Cyber Monday, caching and site simplification (including simplified search functionality) can be a powerful strategy to eliminate all non-revenue-generating hits at the origin in times of peak traffic. It’s a great example of a feature that needs to be toggled quickly, which makes it an ideal scenario for us to use as a test case.
Here are three steps to create a toggle that can almost instantaneously simplify search functionality, using Akamai Edge Redirector and a DNS chain:
1. Set up Your DNS
The most important thing here is to make sure your DNS chain includes a CNAME record with a “key” that tells Akamai if Edge Redirector is on or off; in other words, the DNS chain will be your feature toggle. The DNS record “key” will then CNAME to your expected destination.
Let’s look at an example. At the moment, the www.akamai.com DNS chain looks like this: www.akamai.com. 300 IN CNAME www.akamai.com.edgekey.net. www.akamai.com.edgekey.net. 16727 IN CNAME www.akamai.com.edgekey.net.globalredir.akadns.net. www.akamai.com.edgekey.net.globalredir.akadns.net. 1141 IN CNAME e1699.dscx.akamaiedge.net. e1699.dscx.akamaiedge.net. 16 IN A 18.104.22.168
To prepare for a feature toggle here we will introduce an intermediate record, cdn.akamai.com, that points to either s1.akamai.com (s for search and 1 for true) or s0.akamai.com (s for search and 0 for false). You could directly CNAME www.akamai.com to either s1 or s0, but I like having something in the middle that can be reused to scale it for additional options or new toggles.
When search is on, it will point to s1 and it will look like this: www.akamai.com. 86400 IN CNAME cdn.akamai.com. cdn.akamai.com. 300 IN CNAME s1.akamai.com. s1.akamai.com. 300 IN CNAME www.akamai.com.edgekey.net. www.akamai.com.edgekey.net. 16727 IN CNAME www.akamai.com.edgekey.net.globalredir.akadns.net. www.akamai.com.edgekey.net.globalredir.akadns.net. 1141 IN CNAME e1699.dscx.akamaiedge.net. e1699.dscx.akamaiedge.net. 16 IN A 22.214.171.124
When you want to enable a redirect away from search, point to s0 instead. It will look like this: www.akamai.com. 86400 IN CNAME cdn.akamai.com. cdn.akamai.com. 300 IN CNAME s0.akamai.com. s0.akamai.com. 300 IN CNAME www.akamai.com.edgekey.net. www.akamai.com.edgekey.net. 16727 IN CNAME www.akamai.com.edgekey.net.globalredir.akadns.net. www.akamai.com.edgekey.net.globalredir.akadns.net. 1141 IN CNAME e1699.dscx.akamaiedge.net. e1699.dscx.akamaiedge.net. 16 IN A 126.96.36.199
2. Update Your Configuration
After you set up your DNS, there are three updates you must do to your configuration:
a. Create two variables to track the DNS chain and set the toggle ON or OFF. The toggle will be OFF by default. See image further down for an example.
b. Have professional services enable DNS CNAME tracking on your configuration using advance metadata. Set a TTL of 5s to allow you to make changes in five seconds without having to define a low TTL on your own DNS record (the www.akamai.com example above uses 300 seconds).
This is a one-time configuration step where Professional Services is needed. You will have full control of the toggle thereafter.
c. The third and final configuration change consists of matching on the variable and enabling Edge Redirector.
3. Safe Testing
Finally, you must test the functionality without impacting customer traffic to confirm that the implementation will work when needed. I recommend adding some debug headers with your variables value. Ideally, these headers will be visible only under a header match or for your IPs, as in this screenshot:
Here are some approaches for safely testing your feature toggle:
- Use a different domain: You could use a lower-level environment domain (dev, QA, etc.) to perform the tests by changing the DNS record to s0. The downside to this approach is that you are not testing the production domain and the lower-level environment might not be in the same configuration.
- Test during off-peak hours: This will provide you the most accurate representation of a production cutover, with the only downside being the possible negative impact on users visiting the site during the off-peak hours.
- Limit the scope to only your testers: You can wrap your condition under the same conditions as the debug headers to enable the toggle just for your testers. It will look something like this:
So that’s it. Happy toggling!
Manuel Alvarez is an enterprise architect at Akamai Technologies.