Curse of the living OCSP requests

With Hallowe’en on Tuesday, gather round if you dare and listen to our spoooky story of OCSP requests and Webpagetest. These were… THE RESPONSES THAT REFUSED TO BE STAPLED.

It all started one bright, sunny morning, when we observed closely these mysterious lines in our waterfall report:

Yes, our first view response times were being haunted by OCSP requests. These were easily adding a whole second to our benchmarks.

(For the uninitiated, OCSP is a method for web browsers to ensure that TLS certificates have not been revoked by a certificate authority. The browser will talk to a server managed by the authority itself, and receive a signed response saying whether a given certificate is still valid. There are some concerns with OCSP, and Chrome has apparently chosen to disable it in favour of other mechanisms, but not for Extended Validation certificates.)

It was time to put a stake through the heart of this problem, and implement OCSP Stapling. The server can fetch and cache the OCSP response itself, and send it along with the certificate when a browser visits. This saves a request to the OCSP server.

So we modified our nginx config, and we double-checked it, and third-party tools said that all was well. But then…

…there’s still an OCSP request on first view. (Cue blood-curdling scream.)

Squinting closely at the two waterfalls, we noticed that the first was making requests to “g2.symcb.com” and then “gm.symcd.com”. The second only requests “g2.symcb.com”. This was a clue - after inspecting our certificates closely in the browser, we worked out that the remaining request was actually for our intermediate certificate.

So OCSP Stapling was working as intended - it turns out that there is no way currently to staple both OCSP responses; you can return only one. (There is RFC 6961, but I am not aware of any implementations.)

In practice, you just have to hope that browsers have already cached your intermediate certificate validity elsewhere - they are used by lots of different sites on the web, so this is quite possible.

Another solution would be to obtain a site certificate directly from a root CA, but these are quite rare nowadays, for good security reasons.

Happy Hallowe’en!