Encrypting the Web for All: The Need to Support TLS SNI in the Remaining Clients

by Erik Nygren
919

HTTPS is a protocol for secure communication, where protection, privacy, and integrity of exchanged data are paramount. You’ve used HTTPS sites for shopping, email, login credentials, and anytime it’s important to encrypt data. But more and more traditional web sites that typically weren’t using encryption are now switching to HTTPS for the protection it offers.

Up until recently, getting sites on HTTPS typically encountered two scaling issues due to the nature of the Transport Layer Security (TLS) protocol, which underlies HTTPS: getting certificates and getting dedicated IP addresses for each certificate.  More readily available certificates via Let’s Encrypt have gone a long ways towards helping with the former, and the substantial growth in client support for TLS Server Name Indication (TLS SNI) has removed the need for the latter, as discussed in more detail in my previous blog post.

In particular, the past few years have seen a dramatic increase in client support for TLS SNI, a technology standard that makes HTTPS much more scalable by eliminating the need for dedicated IP addresses per certificate. While early 2014 saw fewer than 85% of requests being sent by clients supporting TLS SNI, a majority of Akamai HTTPS customer configurations now see client TLS SNI usage exceeding 99.6% (when excluding error responses delivered to bots and attacks).

Moreover, for many Akamai products and end-user geographies, a majority of customers see TLS SNI usage for successful requests exceeding 99.8%. High rates of TLS SNI usage are seen across Akamai products and geographies, with only China having a lower rate of TLS SNI usage (likely due to prolonged usage of older clients such as Windows XP, as well as due to some Chinese search bots not sending TLS SNI).

This shift means that deploying HTTPS sites as SNI-only makes sense in most cases, with exceptions for when custom clients or apps are being used that don’t support TLS SNI.

While few web sites were willing to switch to SNI-only and abandon 10% or 20% of their user-base, a larger portion of sites may be willing to break 0.4% or fewer of their clients, especially if many of these are old or insecure clients, bots, or malware. Because TLS SNI has become sufficiently pervasive among most clients, an increasing number of Akamai TLS delivery offerings default to SNI-only offerings which only work for clients supporting TLS SNI.

Which Clients Haven’t Changed to TLS SNI?

As Windows XP has faded away, and as clients which only support SHA-1 certs have stopped working with HTTPS, there is no longer a single major source of clients lacking TLS SNI. The answer to “which clients that I care about will I break by going to SNI-only” now ends up being specific to a given site and the ecosystem of non-browser clients accessing that site. For the majority of sites this isn’t a problem.

Looking at the remaining User-Agents not sending TLS SNI to Akamai, most appear to fall into the following categories.

  • Modern clients that are likely behind a TLS Man-in-the-Middle proxy which may be otherwise downgrading TLS connection security. (US-CERT recently released this advisory on how poorly-implemented HTTPS Interception weakens TLS security and a recent paper with additional details, showing how some don’t even check server certificate identity.)  It is likely that modern client User-Agents also include bots and malware that are spoofing the User-Agent strings and are pretending to be other clients.
  • Remaining Windows XP (and even more ancient Windows) clients. These users are running a past end-of-life operating system and many likely have security issues. It’s unclear how some of these still connect at all following the SHA-1 EOL, but it’s possible that some of the remainder are also behind TLS Man-in-the-Middle proxies that don’t check certificates. We also see some residual traffic from Android 2.x clients which lacked SNI support, but there is less of it than Windows XP and it shows up on a smaller number of customer sites.
  • Some bots and crawlers, including for some search engines. We’ve reached out to a number of those at the top of the list and have seen some deploy TLS SNI support over the past few months. It’s likely that some of these also include a range of malicious and benign bots that are scanning all the entire IPv4 address space for HTTPS servers. At least two search bots based in China notably do not yet support TLS SNI, although another appears to have implemented TLS SNI earlier this year.
  • Various clients and bots implemented on older WebKit forks (such as PhantomJS 1.x), older versions of Java (version 6 and previous, especially with Apache-HttpClient), older versions of curl, and older versions of python. Clients using newer versions of these libraries and languages may also have issues if they don’t initialize TLS connections properly. These span the gamut from malware to headless browsers used for site performance analytics.
  • Custom applications, such as download managers, apps for some gaming consoles, and some mobile clients. These tend to be isolated to individual customer sites.

Before adding redirects on a formerly HTTP-only site to point to HTTPS, it already makes sense to test that clients specific to the site (apps, API clients, testing agents, and the like) properly implement HTTPS. Testing that they send TLS SNI just makes sense to include as an additional test on this front.

Note that it is particularly important to also make sure that clients properly validate server certificates. Research over the years, such as The Most Dangerous Code in the World: Validating SSL Certificates in Non-Browser Software and Danger is My Middle Name:

Experimenting with SSL Vulnerabilities in Android Apps point to many apps not properly validating server certificates, meaning that HTTPS is not providing those apps with security benefits. There is likely some overlap with clients that don’t send TLS SNI and those which don’t properly validate server certificates.

Testing Clients for TLS SNI support

To help test whether custom clients support TLS SNI, I’ve created a simple tool at https://check-tls.akamaized.net/ that will return whether TLS SNI was sent, as well as whether the client negotiated using a modern version of TLS (i.e., TLS 1.2 or soon TLS 1.3). An image, RSS feed, and JSON file are also available there to help with testing different types of apps and clients.

For example, when reading this blog post, these are the results for your browser:

Check as to wheather TLS SNI is present.Different results will be returned based on how the client connects:

Conclusion

As more sites switch to SNI-only, operators of the remaining non-SNI clients will continue to see increasing pressure to implement and deploy TLS SNI. This quickly becomes a virtuous circle, as a shrinking non-SNI client base will encourage increasing use of SNI-only by larger and larger sites over the coming years, especially as historically HTTP-only sites switch to HTTPS.  As such, authors of client software such as bots, crawlers, API clients, and mobile apps should implement TLS SNI support as soon as possible.

Some related links:

Categories: Technology

Suggested Article

LEAVE A REPLY