Shipping an App with App Transport Security

In iOS 9 and OS X 10.11, Apple introduced App Transport Security (ATS), a low-level set of restrictions on apps’ network connections. One of the most visible of these restrictions is the requirement that apps no longer make connections over plain HTTP; instead, the OS enforces the use of HTTPS unless explicitly told otherwise.

There’s already been lots of great discussion about how ATS works – see, for example, Neglected Potential’s Working with Apple’s App Transport Security. Apple has also provided a descriptive tech note on the feature, clearly documenting the expectations of ATS and the exceptions that remain available to developers. And the community has noted in several articles that turning ATS off entirely is generally a Bad Idea.

This article is aimed at a different purpose: to look at the different speed bumps that can show up while building an app alongside ATS, and to explain how to get around them. There are lots of great little tricks that have only cropped up in OS X release notes or on Stack Overflow, or that can only be discovered by building a sample app. Let’s start by digging into where ATS applies.

What App Transport Security Does

ATS is implemented below the Foundation-level networking classes. Since the most common approaches to performing HTTP(S) communication in apps – NSURLSession and the soft-deprecated NSURLConnection – both live in Foundation, ATS will apply to both of them. This means, for example, that it’s not possible to circumvent ATS by continuing to use NSURLConnection. However, lower-level networking APIs in the CFNetwork framework are not subject to ATS’s restrictions.

Correction: the previous paragraph used to claim that CFNetwork-level APIs like CFStreamCreatePairWithSocketToHost() couldn’t circumvent ATS. This is incorrect; CFNetwork provides a way to make secure connections without being subject to ATS’s restrictions, though without the conveniences of the higher-level frameworks. Thanks to @al45tair and @justkwin for pointing out my error on Twitter. See this hastily-assembled sample project for an example of how to make this sort of connection.

ATS also represents an extra layer of security for network connections. It doesn’t replace any existing security or authentication checks that your app might be responding to. This means, for example, that using ATS doesn’t remove the need for methods like NSURLSessionDelegate’s -URLSession:didReceiveChallenge:completionHandler: – any challenge/response communications or certificate validation will still need to be done separately from ATS.

For certain configurations, this means that successfully establishing a HTTPS connection requires some ATS work and some code in an NSURLSession’s delegate. Let’s take a look at such a setup now.

Self-Signed Certificates

Many HTTP servers offer HTTPS, but haven’t necessarily obtained the server certificate from a certificate authority (CA). Instead, these servers might use a self-signed certificate – one that’s been “issued” by that server to itself. Apps that want to connect to that server would then need to trust the server’s word, so to speak, rather than rely on a chain of trust back to a known CA.

The reasons for using a self-signed certificate vary widely, and in lots of circumstances, the correct path forward is to fix the server’s HTTPS configuration using a certificate from a CA. However, developers can’t always guarantee that’s possible – OS X Server, for example, uses a self-signed certificate for its services by default, and customers may be connecting your app to their own self-signed server.

In such a situation, we’d need to first disable ATS – either for one specific server, or for all connections. Let’s assume that users can configure the app to connect to multiple servers, and provide a general ATS exception in Info.plist:

<key>NSAppTransportSecurity</key>
<dict>
    <key>NSAllowsArbitraryLoads</key>
    <true/>
</dict>

However, that’s not quite enough to get a connection going to a server with a self-signed certificate. We’ll also need to inform the networking code in the app that it’s OK to have an invalid certificate. Assuming we’re using NSURLSession, we can do this with a delegate method:

func URLSession(session: NSURLSession, didReceiveChallenge challenge: NSURLAuthenticationChallenge, completionHandler: (NSURLSessionAuthChallengeDisposition, NSURLCredential?) -> Void) {
    completionHandler(.UseCredential, NSURLCredential(trust: challenge.protectionSpace.serverTrust!))
}

Similar behavior is available for other networking technologies, as well: NSURLConnectionDelegate provides a method -connection:willSendRequestForAuthenticationChallenge:, and Core Foundation exposes several kCFStreamSSL* properties that can be set on CFStreams for SSL connections.

Balancing Security with Usability

While the ATS exception and NSURLSessionDelegate method implementation described above will let our app connect to any self-signed HTTPS server, it also represents a fairly significant drop in the security level for our app. The ATS exception effectively disables all the extra checks that iOS 9 and OS X 10.11 use by default for secure connections, leaving our app no better than it would have been under an older OS. Always think carefully before adding ATS exceptions.

Thankfully, we can bring back some of that security. In the hypothetical case above, we said our users might connect our app to any old self-signed server. However, apps often connect back to the developer’s servers (for content, updates, or feedback). We can take advantage of ATS for those connections by defining an additional exception for our domain and all its subdomains:

<key>NSAppTransportSecurity</key>
<dict>
    <key>NSAllowsArbitraryLoads</key>
    <true/>
    <key>NSExceptionDomains</key>
    <dict>
        <key>example.com</key>
        <dict>
            <key>NSExceptionAllowsInsecureHTTPLoads</key>
            <false/>
            <key>NSIncludesSubdomains</key>
            <true/>
        </dict>
    </dict>
</dict>

With this more extensive ATS dictionary in our app’s Info.plist, we turn the “allows insecure HTTP loads” flag back off for the example.com domain and all its subdomains. When our app makes any connection to a host in that domain, then, we’ll be receiving the full protection of ATS.

This exception mechanism can handle even more complex scenarios. For example, consider what might happen if most servers in example.com were correctly secured with HTTPS, but one server – say, insecure.example.com – wasn’t yet fully ATS-compatible. We can handle this by defining another more specific exception for just that host:

<key>NSAppTransportSecurity</key>
<dict>
    <key>NSAllowsArbitraryLoads</key>
    <true/>
    <key>NSExceptionDomains</key>
    <dict>
        <key>example.com</key>
        <dict>
            <key>NSExceptionAllowsInsecureHTTPLoads</key>
            <false/>
            <key>NSIncludesSubdomains</key>
            <true/>
        </dict>
        <key>insecure.example.com</key>
        <dict>
            <key>NSExceptionAllowsInsecureHTTPLoads</key>
            <true/>
        </dict>
    </dict>
</dict>

While it’s not yet documented, ATS exceptions can in fact be nested in this way. Any exceptions specified without the NSIncludesSubdomains key will be evaluated before exceptions with the key, meaning that domain-specific exceptions can override “wildcard” any-subdomain exceptions. (As with all undocumented behavior, this is subject to change in future OS releases.)

Debugging ATS Failures

In the course of building an app with ATS, it’s possible – even likely – that one particular connection fails to pass the ATS checks, and that you’re not sure why. Thankfully, there are several different debugging utilities available for diagnosing ATS problems.

First, let’s look at what an ATS error looks like. Most ATS failures will present as CFErrors with a code in the -9800 series. These are defined in the Security/SecureTransport.h header, available on both iOS and OS X. For example, our self-signed server from before would present the following error without an ATS exception:

2015-08-23 06:33:53.058 SelfSignedServerATSTest[3787:682937] CFNetwork SSLHandshake failed (-9824)
2015-08-23 06:33:53.060 SelfSignedServerATSTest[3787:682937] NSURLSession/NSURLConnection HTTP load failed (kCFStreamErrorDomainSSL, -9824)

And we’d get a similar error (but with a different code) after adding the ATS exception, but before implementing the NSURLSessionDelegate method to allow self-signed certificates:

2015-08-23 06:34:42.700 SelfSignedServerATSTest[3792:683731] NSURLSession/NSURLConnection HTTP load failed (kCFStreamErrorDomainSSL, -9813)

Using CFNetwork Diagnostics

In both of these error cases, we can set the environment variable CFNETWORK_DIAGNOSTICS to 1 in order to get more information on the console about the failure. After turning this variable on in our app’s scheme, you’ll notice a new log line with a path to a diagnostic file; this file, in turn, is filled with information about all the actions the CFNetwork layer is taking on behalf of your app’s networking code. From this file, we can pull more detailed logs like:

Aug 23 06:36:12  SelfSignedServerATSTest[3798] <Notice>: CFNetwork Diagnostics [1:11] 06:36:12.272 {
                   Did Fail
                     Loader: <CFURLRequest 0x7feffae31ab0 [0x1039dc7b0]> {url = https://localhost, cs = 0x0}
                      Error: Error Domain=kCFErrorDomainCFNetwork Code=-1202 "(null)" UserInfo={_kCFStreamPropertySSLClientCertificateState=0, kCFStreamPropertySSLPeerTrust=<SecTrustRef: 0x7feffad2be40>,
                                0 : <cert(0x7feffae357d0) s: markkula i: markkula>
                             )}}
        init to origin load: 0.00655901s
                 total time: 0.026952s
                total bytes: 0
        } [1:11]

While this error doesn’t include a description, we see that the underlying error code is -1202 rather than -9824. This particular code comes from the CFNetwork/CFNetworkErrors.h header, and its name clearly describes the problem: it’s the error kCFURLErrorServerCertificateUntrusted. From there, we could work to fix the server’s HTTPS configuration, or add the necessary ATS exception.

If the log still doesn’t contain quite enough useful information for your purposes, the value for CFNETWORK_DIAGNOSTICS goes all the way up to 3, producing increasingly verbose output at each level. Just be careful – at its most verbose, the diagnostic output could contain sensitive information about the connections being established.

Using nscurl

While CFNETWORK_DIAGNOSTICS can be quite handy, its output can also be somewhat cryptic, and it’s a bit of a hassle to pull the log file off an iOS device. If you have access to an OS X machine running 10.11 or later, the command-line utility nscurl provides some basic ATS debugging capabilities. Simply open a Terminal and run:

nscurl --ats-diagnostics https://example.com

The tool will run through several different combinations of ATS exceptions, trying a secure connection to the given host under each ATS configuration and reporting the result. The ATS configuration changes made by nscurl include:

Turning on the “allow arbitrary loads” flag
Dropping the minimum required TLS version to 1.1, then 1.0
Removing the Perfect Forward Secrecy (PFS) requirement

Including the --verbose flag alongside --ats-diagnostics causes nscurl to also output the exact ATS configuration dictionary being used for each connection, as well as the error returned in its internal NSURLSessionDelegate implementation. For a self-signed certificate on localhost, we might see output like:

Allow All Loads
ATS Dictionary:
{
    NSAllowsArbitraryLoads = true;
}
2015-08-23 06:54:12.625 nscurl[3838:696431] NSURLSession/NSURLConnection HTTP load failed (kCFStreamErrorDomainSSL, -9813)
Result : FAIL
Error : Error Domain=NSURLErrorDomain Code=-1202 "The certificate for this server is invalid. You might be connecting to a server that is pretending to be “localhost” which could put your confidential information at risk." UserInfo={NSURLErrorFailingURLPeerTrustErrorKey=<SecTrust 0x7f8050c7f2b0 [0x7fff78cfe890]>, NSLocalizedRecoverySuggestion=Would you like to connect to the server anyway?, _kCFStreamErrorDomainKey=3, _kCFStreamErrorCodeKey=-9813, NSErrorPeerCertificateChainKey=(
    "<SecCertificate 0x7f8050c7def0 [0x7fff78cfe890]>"
), NSUnderlyingError=0x7f8050c95e10 {Error Domain=kCFErrorDomainCFNetwork Code=-1202 "(null)" UserInfo={_kCFStreamPropertySSLClientCertificateState=0, kCFStreamPropertySSLPeerTrust=<SecTrust 0x7f8050c7f2b0 [0x7fff78cfe890]>, _kCFNetworkCFStreamSSLErrorOriginalValue=-9813, _kCFStreamErrorDomainKey=3, _kCFStreamErrorCodeKey=-9813, kCFStreamPropertySSLPeerCertificates=(
    "<SecCertificate 0x7f8050c7def0 [0x7fff78cfe890]>"
)}}, NSLocalizedDescription=The certificate for this server is invalid. You might be connecting to a server that is pretending to be “localhost” which could put your confidential information at risk., NSErrorFailingURLKey=https://localhost/, NSErrorFailingURLStringKey=https://localhost/, NSErrorClientCertificateStateKey=0}

This tells us that even with arbitrary loads enabled, nscurl doesn’t trust the certificate being used by localhost – we’d need the NSURLSessionDelegate method implementation above to work around this issue. More importantly, it tells us the underlying -1202 error code right up front, rather than requiring that we dig through a CFNetwork diagnostic log to figure out what’s going on.

Wrapping Up

All told, ATS represents a significant step forward in iOS and OS X network security. It helps encourage developers (and, indirectly, users) to transmit data securely across the Internet, regardless of what framework an app is using to perform its network operations. At the same time, ATS provides a flexible system for opting out when some parts of an app aren’t quite ready yet.

Networking is a pretty ubiquitous need in modern apps, and Apple has risen to meet the challenges of developing those apps with a robust set of options for working with and debugging ATS. Between runtime NSErrors, low-level framework logging, and external utilities, plenty of information is available to developers struggling with connections prevented by ATS.