Blog

Dyn (DynDNS) DDoS Attack

 

"Probably the DDoS attack that was noticed and affected the largest number of people ever."

Executive Summary

On Friday, October 21st, millions of users in North America and across the world experienced connectivity issues with many prominent sites, like Twitter, PayPal, Spotify, AWS and more. This was due to a very large attack that common DNS provider Dyn was experiencing, and it is already being considered one of the most prominent attacks. According to Dyn, the attack started 11:00 and 17.

Dyn, also known as DynDNS, is a very large DNS provider that caters to multiple prominent customers (PayPal, Netflix, SalesForce, Deutsche Telekom, TripAdvisor, LinkedIn and more). The first alert that Dyn provided on its status page was at 11:10 UTC. The Dyn DDoS attack impacted both its DNS service and its advanced service monitoring. The attack continued until at least 19:00 UTC, spanning a total of eight hours and comprising three major strikes.

Many websites that rely on Dyn, including Amazon, PayPal, Twitter and GitHub, were reported as being affected. See the table below for more details. An organization affected by this can see short- and long-term recommendations specified below.

According to some websites [1], it has been confirmed that the botnet behind the attack is the Mirai botnet, which became famous recently as it was also used to attack the KerbsonSecurity blog. The Mirai botnet is known for its usage of IoT devices (at least partially), which are the reason for its enormous power.

Customers that were impacted

Here is a list of customers that were impacted by the Dyn DDoS attack.

The following sites were reported to be affected by the site:

  
CustomerNotificationSource

AWS

"AWS reported small-scale connectivity issues to small number of end-points. The cause was "an availability event that occurred with one of our third party DNS service providers. We have now applied mitigations to all regions that prevent impact from third party DNS availability events."

AWS Status Page

PayPal

Alert - Major Impact to PayPal APIs and PayPal Website.

PayPal Merchant Notification Page

Twitter

Issues reported on 21st for all dev services/API.

Twitter Dev API Status Page

GitHub

19:36 BST: We are investigating DNS resolution issues for GitHub.com. 21:35 BST We have migrated to an unaffected DNS provider. Some users may experience problems with cached results as the change propagates.

GitHub Staus Page
  
CustomerNotificationSource

AWS

"AWS reported small-scale connectivity issues to small number of end-points. The cause was "an availability event that occurred with one of our third party DNS service providers. We have now applied mitigations to all regions that prevent impact from third party DNS availability events."

AWS Status Page

PayPal

Alert - Major Impact to PayPal APIs and PayPal Website.

PayPal Merchant Notification Page

Twitter

Issues reported on 21st for all dev services/API.

Twitter Dev API Status Page

GitHub

19:36 BST: We are investigating DNS resolution issues for GitHub.com. 21:35 BST We have migrated to an unaffected DNS provider. Some users may experience problems with cached results as the change propagates.

GitHub Staus Page

Attack Analysis

The main source of information about the Dyn DDoS attack is Flashpoint, which has investigated the attack for Dyn [1]. Here are the highlights.

  • "While the attacks were still ongoing, Flashpoint was able to confirm that at least one portion of the attack was initiated by a Mirai Command and Control server."
  • "Mirai botnets were previously used in the DDoS attacks earlier this month against the ‘Krebs on Security’ blog and the French internet service and hosting provider OVH."
  • "Despite public speculation, Flashpoint assesses with a moderate degree of confidence that the perpetrators behind this attack are most likely not politically motivated, and most likely not nation-state actors."

Red Button recommends taking this data with some caution, and especially to consider that whatever is published must protect a site’s customer reputation.

Open Questions

  1. What were the Dyn DDoS attack vectors and volume?
  2. Was Dyn targeted directly or was it one of its customers?
  3. What does Dyn use for mitigation? is it using its sheer size or also a DDoS mitigation technology? If so, which?
  4. Why did the Dyn DDoS mitigation fail? Was it more because of the attack size or because of a lack of effective mitigation? In other words, did Dyn "screw up" or was it really a groundbreaking attack?

Recommendations

Red Button’s recommendation is to not rely on a single DNS provider. De Facto, our customers that used a second provider in addition to Dyn have weathered the attack without impact.

Following is the recommended setup, provided by order of DDoS resiliency. We understand that for some organizations these recommendations are easier said than done, so we have also included possible caveats and workarounds under the 'Considerations' section below.

  1. Two active DNS servers
    Use two DNS servers that both publish in "active-active" mode.
  2. Use two DNS servers, one as primary and another as backup
    The secondary server will be used only if the primary fails.
  3. Game plan 
    If you cannot set a secondary DNS server, develop and define on your own what your actions would be if your DNS server failed and you hadn’t yet acquired a secondary one. For example, you can use your own DNS server or quickly acquire a different DNS service.

 

For more information, recommendations of DNS vendors and emergency support, contact us here.

History of DNS Attacks & Technical Details

DNS DDoS attacks are extremely common and have become the "weapon of choice" of hackers for several reasons:

  1. DNS service is a point of failure for Internet services. When you take down a DNS server, you take down all the services which are dependent upon it.
  2. DNS is UDP based. It allows spoofing, has modest resources to generate attacks due to connection less protocol, and allows for an attack amplification technique - 1Mbps of attack traffic can end up becoming 100Mbps reflected on the victim.
  3. DNS DDoS mitigation technologies are not as mature and proofed as HTTP DDoS mitigation; in other words, it is harder to stop DNS attacks.

In the past, organizations have maintained their own authoritative DNS servers, but over the years many have chosen to migrate to external DNS service providers like Dyn. DDoS was one of the reasons for this migration - they were simply unable to mitigate these attacks and "outsourced" this problem to someone else.

As a result of this transition in the last few years, DNS providers are handling extremely large and complicated attacks. In some cases, they accept a new customer that is under an ongoing attack, although some providers have also rejected such customers.

There is no question that DNS providers are handling attacks much better than the end customer is. However, this also comes with a risk - if they are unable to do so, their entire customer base goes down.

Snapshots of Affected Sites By Dyn DDoS Attack

Amazon Snapshot

Twitter Snapshot

PayPal Snapshot

GitHub Snapshot

Sign up for our mailing list

 be the first to see DDoS threat alerts, tips, and recommendations.

[mc4wp_checkbox]


You are giving Red Button permission to contact you via email (you can unsubscribe at any time). Read our privacy statement for more information.