On May 5th at 9:43 p.m. there was a crunch in the German part of the Internet: Anyone who tried to access addresses with the ending .de only received an error message – at least if the DNS server used validated DNSSEC signatures. It wasn’t until after 1 a.m. that the problem was completely resolved. DENIC eG, which manages these domains, is responsible for the DNSSEC configuration of the .de zone. This has now provided explanations for the problems.
Read more after the ad
DENIC’s blog post shows that the DNS servers use the Knot software, an open source server service maintained by the Czech domain management organization CZ.NIC, administrator of the .cz domain. At DENIC, Knot runs together with “in-house developments in conjunction with hardware security modules (HSMs)”. In April 2026, this infrastructure was converted to the third generation.
Key collision
The regular exchange of the zone signing key began on May 2nd. A new public key with the ID 33834 was published – 3 days before it was used for signing for the first time. What no one suspected was that there was an error in the self-developed part of the code that resulted in three key pairs with the tag 33834 being generated on the servers, but only one public key was published under this number. When this key was first used to sign SOA entries, the peculiar error pattern arose, which can be traced with a tool like dnsviz.com: Only some of the six DENIC name servers used the correct private key that matched the public key, the others always delivered invalid signatures. The SOA record is changed frequently because every time there is a change to the zone, the serial number in this record changes. This fits with the back and forth that we have already observed in a detailed analysis of the available data.
Those responsible for DENIC note that a code section in the self-built software “was not completely covered by the test scenarios and was therefore not recognized as defective either during test runs or in ‘cold’ parallel operation before commissioning” and further emphasize that three testing tools were used on the zone, all of which were effective – “but the messages were not processed correctly.”
Gambling: With every change to the zone, the serial number changes and a new signature is required. The error caused the six servers to use one of three keys that were under the same ID. Only one matched the public key.
The explanation does not yet provide complete clarity. The code of the in-house developments is not open source, the hardware security module used is not named and the conditions under which the key collision occurred exclusively in the productive system and not in test environments are not explained. After all, they promise to provide more information when the investigation is completed.
All domains affected
Read more after the ad
In the blog post, DENIC also corrects the statement from the night that only domains where DNSSEC was active were affected. This claim was not consistent with observations during the outage. It is true that in addition to the SOA entry, NSEC3 entries were also affected. NSEC3 prevents an attacker with traffic interference from defeating DNSSEC simply by sending the response “DNSSEC is disabled for this domain.” NSEC3 is a cryptographic proof of non-existence. Without valid NSEC3 entries, the DNSSEC check failed for all .de domains.
Conclusion
The case is not yet closed, as DENIC itself writes. In addition to a precise explanation of the problem (including code at best), DENIC has not yet published a list of planned measures on how such a problem could be identified in the future. Not only operators of DNSSEC infrastructure for domains, but also other TLD operators would benefit from such findings. Problems with key collisions at the level of a top-level domain are nothing new: in 2024, the Russian operator of the .ru TLD had an outage, and a key collision was to blame.
(jam)
