Outage Causes Multiple Website Failures (DynDNS Customers Not Affected)

March 31, 2009

This morning, our monitoring system noticed a problem, a big one. From our global perspective, it appears that many online services, including amazon.com (the store, AWS, and S3), salesforce.com, advertisting.com, and petco.com, had some serious DNS troubles. As many of our readers know, DNS is the glue that binds domain names, like dynect.com to their respective server's IP addresses (204.13.248.122). Without DNS, nothing works; no web, e-mail, voip, IM, file sharing, etc.

The Dynect Team quickly began to analyze the situation by checking the resolution chains for these popular web sites. Our analysis revealed that multiple UltraDNS PDNS (a special class of ultradns server) nodes were failing to respond to all DNS queries. Amazon.com clearly knew about the problem, as our monitoring then detected a change in the delegation for amazon.com from UltraDNS' PDNS nodes to their UDNS nodes at approximately 8:50 am Eastern.

We continued our analysis by checking our global performance trending system - some graphs showing the problems are below.

DNS Outage Affecting Domains 2

You can see more graphs from our monitoring on our flickr photostream.

We speculate that the source of the problem may have been a large scale Denial of Service attack against UltraDNS, or an internal operations problem. When we were able to sucessfully query UltraDNS servers, responses were slow to come back, or largely timed out. The problem began to clear itself up around 10:00 am Eastern, when we saw DNS responses returning quickly again, and our favorite sites coming back online.

About Dynamic Network Services Inc.

Founded in 1998, originally as a free DNS service for the Perl and open source communities, Dynamic Network Services Inc. (Dyn Inc.) has grown quietly and steadily to become one of the Internet's largest and most reliable providers of domain, zone and email solutions, through its innovative and well supported services. Still offering its signature free service, the company today plays a key role in keeping the Internet's DNS infrastructure at large running smoothly and stays faithful to its core values of engineering excellence. Additionally, Dyn Inc. also offers Dynect®, an enterprise class dynamic DNS offering.

DynDNS® and Dynect® are registered trademarks of Dynamic Network Services Inc. All other trademarks are the property of their respective owners.