Why is it always DNS?

by Mikey Pruitt on Feb 4, 2022 12:00:00 AM

You’ve heard the joke, sometimes in haiku form:

It’s not DNS

There’s no way it’s DNS

It was DNS

But why is it always DNS? There are a few reasons.

DNS is a simple concept to understand but extremely difficult to put into practice. DNS is often to blame for massive internet outages because even the most savvy companies and educated minds make mistakes while DNS misconfigurations are unforgiving.

At the same time, DNS issues often don’t manifest themselves as DNS issues. Based on the symptom, DNS is often the last thing you’d expect it to be—this explains the denial stage of the haiku.

There was a comment on Reddit recently that sums this up perfectly:

‍

It takes a lot of DNS know-how to have the ability to look at a non-DNS issue and come to the conclusion that it’s a DNS problem before exhausting every other possible option, which is usually the case.

Notable DNS Outages

Amazon AWS - December 2021

A “glitch” in the internal network caused major outages in Amazon’s US-EAST region (Virginia) after a scaling for more capacity event was triggered. Increased latency and errors for services communicating between these networks, resulted in persistent congestion and performance issues on the devices. There are unverified rumors that the glitch was caused by an orchestrated DNS attack of unknown origins.

Facebook - October 2021

A bad BGP router configuration caused a global cascade of outages removing all Meta/Facebook properties from the internet for nearly six hours. To the rest of the internet, Facebook’s infrastructure IPs were unreachable and DNS names stopped resolving. Essentially a black hole where Facebook used to live.

Akamai - July 2021

The network experienced a global issue relating to edge DNS impacting many internet resources caused by a "software configuration update [which] triggered a bug in the DNS system”. Fidelity, the US Securities and Exchange Commission's document search site, Airbnb, British Airways and others were affected. Most of the sites were back up and running in under an hour.

Cloudflare - July 2020

A bad router configuration on the Cloudflare global backbone in Atlanta started announcing bad routes, rerouting massive traffic volumes to the geolocation overwhelming the hardware. The incident only lasted 27 minutes but affected San Jose, Dallas, Seattle, Los Angeles, Chicago, Washington, DC, Richmond, Newark, Atlanta, London, Amsterdam, Frankfurt, Paris, Stockholm, Moscow, St. Petersburg, São Paulo, Curitiba, and Porto Alegre.

And One More Outage

Fastly - June 2021

A software deployment introduced a bug that could be triggered by a specific customer configuration under specific circumstances. On June 8th, the specific configuration and circumstances collided to produce a “broad and severe” Fastly CDN outage.

This outage was not caused by DNS so why do I mention it? Because Fastly and the affected web properties used the magic of DNS to route around the disruption in less than an hour. DNS for the win! Tell your friends.

Masters of DNS are a rare breed and DNSFilter was founded by one and currently has several on staff (I am not one of them, I’d call myself a DNS dabbler).

When we hear "it's always DNS" it's true, not because the technology is bad. It’s because the practitioners are, no offense, inadequate. DNS is so ubiquitous, and it means people are thrown into it without really knowing what they’re doing. It’s not their fault, DNS touches more than they realize.

But once you realize how intertwined DNS is with the entirety of your IT infrastructure, you start to have more respect for it. And you’ll recognize you need to trust the masters of DNS with your DNS security.

Are You a Master of DNS?

Test your knowledge with our short quiz.

______________________________________________

4 https://twitter.com/Akamai/status/1418271515192270850?s=20
5 https://blog.cloudflare.com/cloudflare-outage-on-july-17-2020/
6 https://www.fastly.com/blog/summary-of-june-8-outage

Topics: Featured

How DNSFilter Stops Zero-Day Attacks: The Invisible Threat Costing Businesses Millions

Imagine waking up to find your company's most sensitive data exposed, your systems locked, and your reputation in tatters. This nightmare scenario isn't just a hypothetical—it's the reality for businesses falling victim to zero-day attacks. In 2021, four zero-day exploits targeting Microsoft Exchange servers affected over 250,000 organizations worldwide, leaving countless systems vulnerable to data theft and ransomware.

How DNS Filtering Stops Ransomware

Ransomware attacks have evolved into one of the most pressing cybersecurity challenges of our time. In these attacks, cybercriminals infiltrate an organization’s network, encrypt critical data, and demand payment—often in cryptocurrency—in exchange for the decryption key. As the frequency of these incidents grows, so do their financial and reputational impacts. From small-to-medium-sized businesses (SMBs) to global enterprises, no one is immune...

Machine-Scale Problem, Meet Human-Scale Solution

Greetings fellow humans! It is now 2025 and while we still don’t have flying cars, we do have self-driving cars—that has got to count for something. Some 2.6 million years ago humans began using tools. Today is a different day because, while we are still using machines as tools, machines have surpassed human ability on three important dimensions: The ability to observe change beyond what is humanly possible, efficacy beyond what is humanly possib...