One bad signature took down major .de sites for three hours

On the evening of 5 May 2026, large parts of the German internet stopped working for users on validating DNS resolvers. Bahn, Spiegel, Amazon's German storefront, DHL, ZDF, Postbank, Sparkassen, Hetzner, Strato, IONOS, GMX and Web.de all became unreachable for users behind Cloudflare, Google Public DNS, Quad9, and clients configured to use validating DoH or DoT resolvers. There is no public evidence of an attack. There was no cable cut. There was no infrastructure failure. There was a single malformed cryptographic signature, served by the registry that operates the .de top-level domain, and the protocol behaved exactly the way it was designed to.

The incident is the cleanest, most documentable example of a class of failure that DNSSEC introduces — registry-side bogus signatures — and that registries, by design, are uniquely positioned to inflict. It is also a useful occasion to compare how the four ccTLD registries that matter most to Nordic operators — DENIC for .de, Norid for .no, Internetstiftelsen (IIS) for .se, and Punktum dk for .dk — actually run their signing pipelines. The findings are concrete enough to act on.

What was actually broken on the wire

The technical signature of the failure is precise enough to read from any validating resolver. From Blackfort Technology's incident analysis, querying via Google Public DNS during the outage returned:

$ dig @8.8.8.8 www.blackfort-tec.de A
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL
; EDE: 6 (DNSSEC Bogus): (RRSIG with malformed signature found for
;   a0d5d1p51kijsevll74k523htmq406bk.de/nsec3 (keytag=33834))

$ dig @8.8.8.8 +cd blackfort-tec.de A    # validation disabled
;; ->>HEADER<<- opcode: QUERY, status: NOERROR
;; ANSWER SECTION:
blackfort-tec.de.   600   IN   A   178.104.208.93

With validation disabled (+cd), the domain resolved correctly. With validation enabled, the response was rejected. The domain itself was fine. The DNS records were fine. The authoritative nameservers responded. What failed was the cryptographic proof of one specific record.

The malformed record was an RRSIG over an NSEC3 resource record covering the hash range that blackfort-tec.de, bahn.de, spiegel.de and an unknown number of other domains happened to fall within. The signature was generated by a ZSK with keytag 33834 using algorithm 8 (RSASHA256). It was syntactically valid — it had the right structure, the inception time 20260505174938 and expiry 20260519191938, the correct fields — but the cryptographic value did not verify against the published DNSKEY.

DENIC re-signed the affected NSEC3 hash range with a different key (keytag 32911). DENIC's official statement records that distribution of the corrected zone began at 00:08 CEST on 6 May, with full operating state restored at 01:15 CEST. Earlier, at approximately 22:33 CEST on 5 May, an attempted fix updated the SOA RRSIG but left the malformed NSEC3 RRSIG in place — that detail matters and is examined later in this piece.

Timeline

The clearest published reconstruction comes from cross-referencing DENIC's own statements, Blackfort Technology's protocol-level analysis, and Cloudflare CTO Dane Knecht's public confirmation of the negative trust anchor deployment.

DENIC's update statement, published at 14:30 CEST on 6 May, attributes the cause to a routine ZSK rollover:

Die Störung steht in einem zeitlichen und logischen Zusammenhang mit einem üblichen regelmäßigen Schlüsselwechsel. Dabei sind nicht validierbare Signaturen erzeugt und verteilt worden. Vorsorglich wurden die künftigen Wechsel ausgesetzt, bis wir die genauen technischen Ursachen ermittelt haben.

Translation: The disruption is temporally and logically connected to a routine regular key rollover. In the process, signatures that could not be validated were generated and distributed. As a precautionary measure, future rollovers have been suspended until we have determined the exact technical causes.

DENIC stops short of asserting causation — only "a temporal and logical connection." According to DENIC's own 2016 article describing their KSK rollover, the .de ZSK is replaced every five weeks. That places the cadence of the operation at roughly ten times per year.

Why one bad signature reached so many users

DNSSEC is a fail-closed protocol. When a resolver validates DNS responses and the validation fails, the resolver does not return what it received with a warning. It refuses to answer at all, returning SERVFAIL. From the resolver's point of view, a malformed signature and a forged signature look identical — both indicate that the chain of trust is broken. Refusing to answer is the only safe behaviour.

This is not a bug in DNSSEC. It is the property the protocol exists to provide. The cost of that property is that an operator who publishes a broken signature themselves causes the same outcome as an attacker who tampers with the records.

The blast radius depends on two things: how widely deployed validating resolvers are, and how high in the DNS hierarchy the broken signature sits.

Validating resolvers are not a niche. Google Public DNS (8.8.8.8), Cloudflare (1.1.1.1) and Quad9 (9.9.9.9) all validate strictly. So do increasing numbers of ISP resolvers, organisational resolvers behind corporate firewalls, and clients configured to use validating DoH or DoT resolvers. The frequently repeated framing — that only 3.6% of .de domains are DNSSEC-signed and therefore the impact was limited — confuses domain count with user impact, and misreads the failure mechanism.

The malformed signature was on an NSEC3 record in the .de parent zone — the cryptographic proof a validating resolver needs to confirm whether a child name is signed (has a DS record) or insecurely delegated (does not). When that proof is unverifiable, the answer is rejected regardless of whether the queried domain operates DNSSEC of its own. The hash range covered by the malformed NSEC3 happened to include high-traffic names — bahn.de, spiegel.de, blackfort-tec.de, and an unknown number of others. Most of them, including Bahn and Spiegel, are not themselves DNSSEC-signed. They were unreachable anyway, because their parent zone could no longer prove their insecure delegation. A user behind Cloudflare's resolver saw SERVFAIL. A user behind a non-validating ISP resolver saw the site load. The resolver, not the domain, decided whether validation happened.

Because the malformed signature sat at the TLD level — one layer below the root — there is no fall-through to a parent that could have caught it. If the chain breaks at any link, validation stops. A single broken NSEC3 RRSIG in the .de zone became, for several hours, an unreachable Germany for any user behind a strict resolver.

The negative trust anchor: turning off security to restore service

Cloudflare's response was to deploy a Negative Trust Anchor (NTA) for .de. NTAs are defined in RFC 7646 (Definition and Use of DNSSEC Negative Trust Anchors), which permits a validating resolver to temporarily stop validating responses for a specific misconfigured zone. The records are still served, but the cryptographic check is bypassed for the duration of the NTA.

This is the operational equivalent of turning off the security feature that is currently producing outages. It is well within the protocol — RFC 7646 specifically anticipates this scenario — but it is worth naming clearly. To restore reachability of .de, Cloudflare suspended the integrity guarantee that DNSSEC was deployed to provide. For the duration of the NTA, a .de domain's response from Cloudflare's resolver was no more cryptographically authenticated than a plain DNS response in 1995.

This is not a criticism of Cloudflare. NTAs are the correct mechanism in this situation. It is a useful observation about the failure mode itself: when the registry produces invalid signatures, the only way validating resolvers can deliver content to users is to stop validating. There is no clever recovery path for the resolver side. It is the registry's bug, and only the registry can fix it. Google Public DNS and Quad9 chose to remain strict. Users on those resolvers saw SERVFAIL until DENIC's re-signing propagated.

What DENIC said, and what they did not say

DENIC's official statement said the incident "stand im Zusammenhang mit der DNSSEC Signierung" and that "fehlerhafte DNSSEC-Signaturen" were distributed. The 14:30 CEST update added the routine-rollover context quoted above. What DENIC has not said publicly, as of this writing, is what their pre-publication validation pipeline looked like at the moment the malformed signature was distributed, why the 22:33 CEST fix attempt updated the SOA RRSIG without updating the malformed NSEC3 RRSIG, and what controls are being added before rollovers resume. Those answers will come in the post-mortem. Until they do, the most useful thing observers can do is examine how comparable registries describe their own pipelines.

A Nordic comparison

Most ccTLD operators publish a DNSSEC Policy and Practice Statement (DPS) following the framework in RFC 6841. These documents describe what the registry commits to in writing. They do not capture every internal control, but they are the authoritative public statement. I compared the .de situation to the published practice of the three Nordic ccTLD operators that matter most to organisations operating from Norway, Sweden, and Denmark.

All section references below are to each operator's own DPS or, where the DPS is not public, to the operator's own published statements. Live values were verified by direct DNS query against 8.8.8.8 on 6 May 2026.

Algorithm and denial of existence

The .de failure was specifically a malformed RRSIG over an NSEC3 record. .no and .dk produce the same kind of record using the same general construction, with the modern parameters from RFC 9276 (Guidance for NSEC3 Parameter Settings) in both cases. .no's published DPS still describes Opt-Out NSEC3 (DPS section 6.2), but the live .no zone serves NSEC3 without Opt-Out — practice has moved beyond the published document. .se uses NSEC. NSEC has different machinery (no hashed denial of existence, no opt-out) and a different set of failure modes than NSEC3.

The algorithm difference (RSA at .de and .se, ECDSA at .no and .dk) does not change the failure class. A logic error in the signing pipeline can produce a malformed signature regardless of the cryptographic algorithm used to generate it.

Rollover and resilience

Two observations matter here. The first is that DENIC operates the highest-frequency ZSK rollover among this group — every five weeks, roughly ten rollovers per year. Each rollover is a discrete failure opportunity. RFC 7583 (DNSSEC Key Rollover Timing Considerations) does not require a five-week cadence; the choice is operational. The second is that .no and .dk are the two registries running on the more modern algorithm (ECDSA) and the more modern NSEC3 parameters (no opt-out, no salt, no iterations), but .no runs on a SoftHSM and a DPS that has not been revised since February 2022, while .dk does not publish a DPS at all.

Signing software family

DENIC has not publicly documented the specific software running its signing pipeline. The three Nordic operators all have a documented or strongly attested link to the OpenDNSSEC project. Norid's DPS section 5.8.1 states explicitly:

The signing system is based on the open source products OpenDNSSEC and SoftHSM.

Internetstiftelsen's published history of .se includes a 2010 announcement of a DPS revision "as a preparation for the implementation of a system for automatic key generation and signing, OpenDNSSEC", and a 2020 blog post on the .se algorithm rollover that describes operating on OpenDNSSEC 1.4.3 at the time. The current public DPS does not name the signer, but does not contradict OpenDNSSEC use either. Punktum dk's signer software is not named in any document I could locate; a 2012 OpenDNSSEC training presentation lists .dk among the registries using OpenDNSSEC at that time, which should be taken as historic context, not current state.

OpenDNSSEC is widely deployed and well-maintained. Naming it is not a criticism — it is a load-bearing piece of internet infrastructure. It is also worth noting that the most directly comparable recent TLD-scale outage, the InternetNZ .ac.nz incident of 29 May 2023, was attributed by APNIC's analysis to OpenDNSSEC behaviour:

This appears to be an outcome of OpenDNSSEC behaviour, where OpenDNSSEC made an incorrect assumption about the intended key state for the zone and stopped using the old KSK to sign the common DNSKEY record, thereby breaking the chain of trust.

The same software family underpins three of four Nordic registries.

The publishing gate

This is the most consequential comparison, and where the practice statements differ most. The DENIC failure was not an invalid SOA, an expired key, or a broken delegation. It was a single malformed NSEC3 RRSIG inside the zone. The Blackfort analysis records that DENIC's first attempted fix at 22:33 CEST updated the SOA RRSIG without updating the broken NSEC3 RRSIG — strongly suggesting that DENIC's own internal verification surfaced an SOA-level signal but not the underlying defect.

A pre-publish gate that re-validates the entire candidate zone the way Google Public DNS or Cloudflare's validators do — every RRSIG, including each NSEC3 RRSIG across the hash space — would have caught the broken signature before distribution. Whether each registry runs such a gate is described differently in each operator's public materials.

Norid (.no). DPS section 6.7 specifies:

Before publishing a signed zone on the name server, the zone must pass a number of checks, examples are: Verification of the chain of trust from the DS record in the parent zone to the signature of the SOA record in the child zone. Verification that the validity period of the signature of the SOA record is at least 2 days in the future.

The verification examples cover the SOA RRSIG and the parent-to-zone chain. DPS section 6.8 adds a generic standards-conformance check ("verifies that all resource records conform with the current standards"). The published practice does not explicitly require comprehensive cryptographic validation of every RRSIG in the zone, including each NSEC3 RRSIG across the hash space, against an independent validator.

Internetstiftelsen (.se and .nu). DPS section 6.6 specifies:

To ensure valid signatures and integrity of the DNSKEY record, a set of checks are automatically run at each signing occasion. These controls include verification of signatures using the Delegation Signer (DS) records registered with IANA for the Root Zone, as well as verification of time and date. Zone information which does not pass the automatic checks will put the production of a new zone file on hold and become flagged for manual intervention and troubleshooting. The production of a new zone file is on hold until the troubleshooting and error-handling is completed.

This is the strongest documented "hold the zone if checks fail" language among the three Nordic registries, but the depth of the resource-record validation is described in general terms ("in accordance with the current standards") rather than as explicit signature-level verification.

Punktum dk (.dk). The most explicit statement of the four operators comes from a registrar newsletter published in February 2023, in the context of a tightening of NSEC3 parameters:

All DNSSEC signatures are validated before each change to the public .dk zone. This validation is CPU heavy, which means that after March 22nd, updates to the .dk zone will happen every 3 minutes instead of every 2 minutes like today.

Punktum dk does not publish a formal DPS, but the operational claim — every signature, every zone update — is unambiguous, public, and tied to a specific resource cost (the slowed cadence). It is the strongest registry-side validation gate language I encountered while researching this piece.

DENIC (.de). What DENIC's pre-publish validation looked like at the moment of the incident is not publicly documented. The promised post-mortem will be the next useful data point.

The recurring class

The .de outage is not novel in kind. The most directly comparable recent precedent is the .nz incident of 29 May 2023, in which InternetNZ's KSK rollover for the .ac.nz second-level zone broke DNSSEC validation for affected domains. The signing system removed the old key too early during the rollover, and the chain of trust broke for users on validating resolvers. The Tor Project experienced an unrelated but mechanically similar .org DNSSEC outage on 20 September 2025, when key rotation happened earlier than planned. Public lists of DNSSEC outages document a steady drumbeat of validation incidents across BIND, Knot, dnsmasq, and signer pipelines.

The pattern is consistent. The protocol is sound. The implementations are mature. The recurrent failure surface is the operational pipeline that produces signatures: key rollovers, signing tool behaviour, pre-publication verification, and the moment of zone distribution.

What this all adds up to

Two propositions stand on the evidence above. They are not in tension.

The first is that DNSSEC behaved exactly as specified during the .de outage. A registry served signatures that did not validate. Validating resolvers refused to answer. Cloudflare deployed an NTA per RFC 7646 to restore reachability at the cost of validation. None of this is a flaw in the protocol. The protocol exists to refuse answers under those conditions, and it refused them.

The second is that the recurrent failure surface for TLD-scale DNSSEC is the registry's own publishing pipeline. The .de incident's first fix attempt updated the SOA RRSIG without correcting the malformed NSEC3 RRSIG, which is a significant operational signal: the visible defect was at the NSEC3 level, but the operator-facing signal that triggered the fix appears to have been at the SOA level. The remedy for this class of failure is not "DNSSEC is broken." It is a publishing gate that validates every RRSIG in the candidate zone, against multiple independent resolvers, before zone distribution — the same validation that strict public resolvers perform after distribution, run earlier in the pipeline.

Of the four registries compared here, Punktum dk is the only one that publishes language committing to validation of every DNSSEC signature on every zone update. Internetstiftelsen describes a hold-the-zone mechanism without committing to full RRSIG-level validation in writing. Norid's DPS gives examples that focus on SOA-level signals. DENIC does not publish a DPS for .de and the .de incident itself shows that whatever validation was in place was not sufficient to catch the malformed signature before it left the signing system.

This is not the same as saying that .no or .se would have produced a comparable outage. The operational track records of all three Nordic registries are clean at TLD scale. But the structural exposure to this class of failure — a malformed signature reaching production — is real for any registry whose published practice does not explicitly validate every signature in the zone before distribution. That is the relevant takeaway for organisations operating on Nordic ccTLDs, and it is the reason DENIC's post-mortem matters for more than just .de.

What organisations on Nordic ccTLDs should actually do

The .de outage is a useful event to act on. Five concrete steps:

Establish DNSSEC monitoring against multiple validating resolvers as part of regular security monitoring. Automated queries from 8.8.8.8, 1.1.1.1, and 9.9.9.9 against your own domains, alerting on SERVFAIL or EDE: 6 (DNSSEC Bogus), are not optional at this point. Detection that depends on customer complaints is too slow.

Monitor RRSIG expiry separately. Expired signatures are a more common cause of self-inflicted outages than the registry-side bogus class examined here, and the same monitoring infrastructure handles both.

Document an incident response that explicitly includes the case where the registry — not your infrastructure — is the root cause. A SERVFAIL from your zone is your problem; a SERVFAIL from the TLD is everyone's problem. The escalation paths are different. Knowing which is which in the first ten minutes matters.

For organisations holding .no, .se, or .dk domains where availability under registry-side DNSSEC failure matters, consider whether a parallel domain on a different TLD is part of disaster recovery. This is not a recommendation to abandon ccTLDs — it is the same logic that puts a backup MX on a different provider.

For organisations subject to NIS2 or DORA, DNS validation status belongs in the same SIEM as certificate monitoring and availability checks. The May 5 outage is a worked example of a critical-infrastructure event that did not announce itself through any traditional alerting surface.

Open questions for DENIC

DENIC's promised post-mortem will be useful. Specific questions worth answering:

How is the candidate .de zone validated cryptographically before distribution? Does that validation include every RRSIG, including all NSEC3 records, or is it scoped to the SOA RRSIG and chain-of-trust set? Why did the 22:33 CEST fix attempt update the SOA RRSIG rather than the malformed NSEC3 RRSIG — does this indicate that the operator-facing validation surfaced the wrong signal? Was there an independent third-party-resolver canary running against a.nic.de through n.nic.de after each rollover, and if so, what did it report between 21:57 CEST and the eventual fix? What is the median and worst-case latency between a signing run and external resolver consistency? And, more broadly: is the published five-week ZSK cadence the right cadence given that each rollover is a discrete failure opportunity?

The answers will tell the rest of the registry community where the operational gap actually was — and let Norid, IIS, and Punktum dk verify that the same gap is not present in their own pipelines.

Closing

A single signature can take a national domain offline for several hours. The protocol behaved correctly. The registry made a single mistake. The mistake was distributable at the speed of zone propagation, and roughly three hours of unreachable Bahn, Spiegel, Postbank and Sparkassen later, the only remediation available was a re-sign by the same operator who produced the bug.

All three Nordic ccTLDs operate signed parent zones where the same class of registry-side publishing failure is, in principle, available. None of them has shipped it. The next post-mortem from DENIC is the document that decides whether their existing controls are enough — for .de, for .no, for .se, for .dk, and for every ccTLD whose architecture rhymes with these.

Sources

Incident

• DENIC blog, Technische Störung bei .de-Domains behoben (6 May 2026) — primary statement from the registry
• Blackfort Technology, DNSSEC Failure in the .de Zone — protocol-level incident analysis with verbatim dig output
• The Register, Denic sorry for DNSSEC error that crashed Germany's internet
• Dane Knecht (Cloudflare CTO), confirmation of NTA deployment per RFC 7646
• Cybernews, Millions of .de websites are unreachable due to DNSSEC failure
• ip.network, Major DNS Outage Hits .de Domains
• Hacker News thread, .de TLD offline due to DNSSEC
• DENIC, DNSSEC: New Hardware, New Key (2016) — source for five-week ZSK cadence

Nordic ccTLD operator practice

• Norid, DNSSEC Policy & Practice Statement for .NO, revision 1e2 (2022-02-01)
• Norid, DNSSEC for .no
• Internetstiftelsen, DNSSEC Practice Statement (DPS), version M (2022-09-06)
• Internetstiftelsen, Lessons learned from the .SE algorithm rollover
• Punktum dk, Registrar newsletter, February 2023
• Punktum dk, DNSSEC
• M. Toft and G. Sluyterman, DNSSEC og OpenDNSSEC (2012)

Class precedents

• InternetNZ, DNSSEC chain validation issue: technical incident report (June 2023)
• APNIC blog, DNSSEC and .nz (April 2024)
• Tor Project status, DNSSEC outage (20 September 2025)
• ianix, DNSSEC outages list

Standards

• RFC 5155 — NSEC3
• RFC 6605 — ECDSA for DNSSEC
• RFC 6781 — DNSSEC Operational Practices
• RFC 6841 — DPS framework
• RFC 7583 — Key rollover timing
• RFC 7646 — DNSSEC Negative Trust Anchors
• RFC 8901 — Multi-signer DNSSEC
• RFC 9276 — NSEC3 parameter guidance
• IANA DNSSEC Algorithm Numbers registry

Live values for .de, .no, .se, and .dk (DNSKEY records, NSEC3 parameters, RRSIG inception and expiry, signing keytags) were verified by direct DNS query against 8.8.8.8 on 6 May 2026.