On Saturday, at 10:48 UTC, Sectigo’s AddTrust legacy root certificate expired, causing a bit of weekend havoc for thousands of websites and services that rely on it for making a secure TLS/SSL connection.
“Generally speaking, this is affecting older, non-browser clients (notably OpenSSL 1.0.x) which talk to TLS servers which serve a Sectigo certificate chain ending in the expired certificate,” wrote Andrew Ayer, founder of SSLMate, in a blog post.
When connecting to a TLS server, the server sends a certificate to the client to establish its identity, and an intermediate certificate that links the server cert to a trusted root certificate. This forms a chain of trust. When that chain breaks – because a certificate is invalid or missing – errors occur.
After the AddTrust External CA Root and the USERTrust RSA CA intermediate certificate expired, applications like Red Hat Enterprise Linux 7, Roku’s streaming media service, and Algolia, started having problems.
Users of the RoboForm password manager found they could not connect to the RoboForm server.
— Peter Pushoofd (@pushoofd) May 30, 2020
Some apps, like website monitoring app Oh Dear, warned users ahead of time so they could remove the expiring certificate before things broke.
GCC 10 gets security bug trap. And look what just fell into it: OpenSSL and a prod-of-death flaw in servers and apps
Ayer’s SSLMate also managed just fine. “I saw this coming over a year ago, and configured SSLMate to start providing a chain without AddTrust External CA Root,” he wrote, in what we imagine was a slightly smug tone.
Ayer has been compiling a list of affected applications and services on Twitter.
The damage as measured in time seems to be not more than a few hours. Heroku was down for about 70 minutes fixing things up. Turnitin reported downtime of about 2.5 hours.
Modern browsers should not be affected because they’re designed to use the SHA-2 root (COMODO or USERTrust) as an alternative trust chain. But applications that rely on older versions of OpenSSL and GnuTLS weren’t designed to deal with bad certs.
“Lots of embedded software doesn’t handle this,” said Ryan Sleevi, a Google software engineer, via Twitter. “OpenSSL was, and is, fundamentally shit at verifying ‘real’ certificates. It has a long history of not coping with the Internet, and only really handling toy/enterprise-specific CAs that are linear. But even then, not very well.”
El Reg readers report that UK-based cert biz Trustico and US-based SSLS.com have been issuing certificates that are suddenly failing because they were issued without checking that all the certs in the chain of trust are valid.
The Register asked SSLS.com for comment, and we’ve not heard back.
University of California, Berkeley has posted a notification to systems administrators outlining potentially affected systems, such as Linux or macOS OpenLDAP clients.
Of particular concern, the university said, are systems and devices that haven’t seen security updates since 2015, such as Apple Mac OS X 10.11 (El Capitan) or earlier, Apple iOS 9 or earlier, Google Android 5.0 or earlier, Microsoft Windows Vista & 7 (if the Update Root Certificates Feature has been disabled since before June 2010), Microsoft Windows XP (if an Automatic Root Update has not been received since before June 2010), Mozilla Firefox 35 or earlier, Oracle Java 8u50 or earlier, and embedded devices (e.g. copy machines) that have not installed a firmware update since before mid 2015.
“In a perfect world, all of your libraries would be up-to-date and you wouldn’t be using clownish TLS implementations like GnuTLS,” wrote Ayer. “But the world isn’t perfect.” ®